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In  January  1990,  the  Food  Safety  and  Inspection  Service  (FSIS)  of 
the  U.S.  Department  of  Agriculture  initiated  efforts  to  determine 
how  to  implement  the  Hazard  Analysis  and  Critical  Control  Point 
(HACCP)  system  in  regulated  meat  and  poultry  plants.  HACCP 
is  a  process  control  system  that  identifies  "critical"  points  in  the 
food  production  process  that  should  be  controlled  to  prevent  food 
safety  hazards  (i.e.,  microbiological,  chemical,  and  physical 
hazards)  from  occurring. 

Agency  activities  included  consultations  and  public  hearings  to 
explain  HACCP  and  the  Agency's  efforts,  workshops  to  develop 
model  HACCP  plans,  in-plant  testing  to  trial  the  specific  plant 
HACCP  plans  at  nine  volunteer  plants,  and  an  assessment  of  the 
overall  experience  of  the  Agency. 

A  series  of  reports  will  be  written  on  the  various  phases  and 
activities  undertaken  by  the  Agency  and  participants  during  the 
study.  This  is  one  of  the  reports.  For  further  information 
concerning  FSIS  HACCP  activities  contact: 

Dr.  Dorothy  Stringfellow 
HACCP  Office 

14th  &  Independence  Ave. ,  SW 
Room  4444 

Washington,  DC  20250 
Phone:  (202)  690-2087 
FAX:  (202)  690-3886 
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SUMMARY 


In  January  1990,  the  Food  Safety  and  Inspection  Service  (FSIS)  began  a  study  to  determine  the 
process  for  Hazard  Analysis  Critical  Control  Points  (HACCP)*  implementation  in  meat  and 
poultry  operations.  As  part  of  the  study,  five  FSIS/industry  workshops  were  held  to  develop 
generic  HACCP  models.  Workshops  were  held  to  develop  HACCP  models  for  Refrigerated 
Foods,  Cooked  Sausage,  Poultry  Slaughter,  Ground  Beef,  and  Swine  Slaughter.  An  industry 
steering  committee  was  formed  after  each  workshop  to  fine-tune  the  generic  model  and  represent 
industry  if  any  changes  needed  to  be  made  in  it  following  testing  in  pilot  plants. 

This  report  describes  the  workshops,  their  steering  committees,  and  the  process  of  adapting  the 
generic  models  from  three  of  the  workshops  to  nine  pilot  plants. 

According  to  the  participants,  useful  generic  HACCP  models  were  produced  at  the  FSIS/industry 
workshops.  The  workshop  format  for  developing  HACCP  models  worked  well.  Eighty-four 
percent  of  the  participants  who  filled  out  questionnaires  said  the  workshop  accomplished  what 
they  expected.  They  also  stated  that  the  workshop  format  was  a  sound  approach  for  producing 
a  generic  HACCP  mode.  When  asked  to  suggest  alternatives  methods  for  developing  generic 
models,  more  than  half  the  participants  volunteered  that  the  workshop  method  was  the  best 
method  for  developing  generic  models. 

The  steering  committees  played  an  important  role  in  the  workshop  process  by  fine-tuning  the 
HACCP  models  so  they  were  consistent  within  themselves  and  with  other  generic  models.  The 
committees  handled  issues  which  were  not  able  to  be  completed  at  the  workshops. 

The  generic  models  developed  at  the  workshops  were  found  to  be  applicable  in  the  volunteer 
pilot  plants.  The  HACCP  Special  Team  requested  that  only  one  change  in  a  generic  model  be 
made  by  a  .steering  committee  based  on  the  plant-specific  plans  developed  in  the  plants.  More 
information  or  training  about  how  to  design  a  HACCP  plan  or  a  more  detailed  generic  model 
would  be  necessary  before  some  plants  could  develop  a  HACCP  plan  on  their  own. 


♦Hazard  Analysis  Critical  Control  Points  (HACCP)  is  a  process  control  system  that  identifies  "critical"  points  in  the  food 
production  process  that  should  be  controlled  to  prevent  food  safety  hazards  from  occurring. 
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I.  INTRODUCTION 


Purpose 

To  examine  the  role  of  joint  FSIS/industry  workshops  in  the  development  of  generic  HACCP* 
models  for  meat  and  poultry  products. 

Background 

In  January  1990,  the  Food  Safety  and  Inspection  Service  (FSIS)  began  a  study  to  determine  the 
process  for  Hazard  Analysis  Critical  Control  Points  (HACCP)  implementation  in  meat  and 
poultry  operations.  As  part  of  the  study,  five  FSIS/industry  workshops  were  held  to  develop 
generic  HACCP  models.  Workshops  were  held  to  develop  HACCP  models  for  Refrigerated 
Foods,  Cooked  Sausage,  Poultry  Slaughter,  Ground  Beef,  and  Swine  Slaughter.  Participants  in 
these  workshops  were  representatives  from  industry,  trade  associations  and  government. 
Workshops  were  open  to  the  public  for  observation,  on  a  reserved  space  basis.  Time  was  allotted 
for  observer  comments,  but  observers  were  not  allowed  to  participate  in  the  model  development 
process. 

The  objectives  of  the  workshops  were  to: 

1.  Assure  participants  understand  the  seven  HACCP  principles  and  their  application. 

2.  Assist  the  industry  participants  in  developing  a  generic  HACCP  model  for  their 
process/product. 

3.  Use  a  workshop  design  which  will  help  capture  input  from  all  participants  in  the 
industry  HACCP  model. 

4.  Provide  HACCP  information  needed  for  the  participants  to  tailor  the  generic  HACCP 
model  into  a  specific  HACCP  plan  for  their  products/processes  at  their  establishments. 

These  objectives  were  to  be  met  by  involving  all  participants  in  the  HACCP  model  design  by 
structuring  the  workshops  in  short  group  participation  segments  for  both  large  and  small  group 
activities. 

The  workshops  were  facilitated  by  a  Special  Team  of  headquarters  and  field  employees  which 
orchestrated  all  workshop  sessions,  prepared  all  needed  materials,  and  led  the  sessions.  Special 
Team  members  were  FSIS  personnel  selected  for  leadership,  communication  skills  and  a  strong 
background  in  FSIS  operations  and  regulations. 

Five  Subject  Matter  Experts  (SME)  were  chosen  competitively  from  FSIS  field  staff  for  each 
workshop.  Their  role  was  to  provide  technical  advice  on  each  specific  workshop  product/process. 
Pre-workshops  were  held  prior  to  each  workshop  with  the  SMEs  to  work  out  potential  problem 
areas  and  plan  workshop  specifics. 

♦Hazard  Analysis  Critical  Control  Points  (HACCP)  is  a  process  control  system  that  identifies  "critical"  points  in  the  food 
production  process  that  should  be  controlled  to  prevent  food  safety  hazards  from  occurring. 


A  workshop  steering  committee  of  industry  participants  was  established  after  each  workshop. 
This  committee  was  responsible  for  refining  the  generic  HACCP  model  for  consistency  and  for 
making  any  changes  to  the  model  following  testing  in  volunteer  plants. 

After  the  workshops  were  held,  three  of  the  workshop-generated  HACCP  models,  Refrigerated 
Foods,  Cooked  Sausage,  and  Poultry  Slaughter,  were  each  tested  in  three  volunteer  plants  for  a 
total  of  nine  plants.  The  volunteer  plants  developed  plant-specific  plans  based  on  the  generic 
HACCP  models. 


Methodology 

The  Policy  Analysis  Unit  (PAU),  FSIS,  as  part  of  the  HACCP  study,  examined  the  workshops. 
A  questionnaire  was  developed  for  workshop  participants  and  observers,  for  steering  committee 
members,  and  for  the  Special  Team  and  pilot  plant  employees  involved  in  adaption  of  the  generic 
model  to  plant-specific  HACCP  plans.  Participants  and  observers  were  asked  questions  related 
to  the  design  and  operation  of  the  workshop,  the  workshop  materials,  the  process  employed  in 
developing  a  generic  HACCP  model,  and  their  views  concerning  the  workshop  approach. 
Steering  committee  members  were  asked  questions  concerning  the  roles  and  tasks  of  the  steering 
committee,  their  time  and  effort,  and  the  cost  to  their  company  of  their  participation.  The  Special 
Team  and  plant  employees  were  asked  questions  assessing  their  time  and  effort  spent  developing 
a  plant-specific  plan  based  on  the  generic  model  developed  at  the  workshop. 

Questionnaires  completed  by  participants  and  observers  were  analyzed  and  reports  were  prepared 
after  each  workshop.  The  same  questionnaire  was  used  for  all  five  workshops  with  minor 
changes  to  address  particular  issues  relevant  to  a  specific  workshop. 

Individual  reports  were  written  for  each  steering  committee  and  an  overall  report  based  on  these 
was  prepared.  In  addition,  individual  reports  were  written  about  the  development  of  a  plant- 
specific  plan  in  each  pilot  plant  and  an  overall  report  based  on  these  was  prepared. 


Report  Plan 

The  first  section  of  this  summary  report  presents  individual  synopsis  of  each  of  the  five 
workshops  including  their  steering  committee  actions  and  plant-specific  adaption  experiences,  if 
applicable.  The  final  section  presents  some  concluding  remarks  about  the  workshops,  steering 
committees  and  plant-specific  adaption  experiences. 
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II.  INDIVIDUAL  HACCP  WORKSHOPS 


Refrigerated  Foods  Workshop 

The  Refrigerated  Foods  Workshop,  the  first  to  develop  a  generic  HACCP  model,  was  held  in 
Baltimore,  Maryland,  on  February  26-28,  1991.  This  workshop  focused  on  "keep  refrigerated" 
products  containing  uncured  meat  or  poultry  that  are  cooked,  assembled,  then  packaged.  A  total 
of  40  participants  and  42  observers  were  in  attendance. 

The  steering  committee  met  in  Washington,  DC  shortly  after  the  workshop  to  "wordsmith"  the 
model  and  to  address  issues  raised  at  the  workshop.  In  addition,  the  committee  discussed  the 
applicability  of  HACCP  principle  #1  concerning  risk  assessment  and  risk  categories. 

The  generic  model  developed  at  the  workshop  was  tested  in  three  pilot  plants.  In  general,  the 
generic  model  was  successfully  adapted  to  plant-specific  plans  in  the  pilot  plants.  However,  all 
three  plants  included  a  critical  control  point  for  "receiving"  in  their  plant-specific  plans  which 
was  not  included  in  the  generic  model.  The  steering  committee  was  contacted  to  change  the 
generic  model. 

Findings 

Ninety-three  percent  of  participant  respondents  agreed  or  strongly  agreed  that  the  workshop 
resulted  in  a  useful  generic  HACCP  model. 

Ninety-two  percent  of  participant  respondents  indicated  that  the  workshop  met  their  expectations. 
Group  discussion,  debate  with  peers,  and  the  open  forum  between  government  and  industry  were 
cited  as  the  most  beneficial  aspects  of  the  workshop. 

Participant  respondents  said  the  written  materials  were  most  useful  for  "developing  assigned 
[critical  control  points]  CCP  monitoring  plan"  and  "developing  verification  procedures."  They 
felt  the  materials  to  "assess  ingredient  hazards"  were  least  useful. 

Participant  and  observer  respondents  had  different  views  on  the  benefit  of  including  observers. 
Seventy-six  percent  of  observers  strongly  agreed  that  "the  presence  of  observers  was  beneficial 
to  the  workshop."  Only  53  percent  of  the  participants  did. 

Steering  committee  members  stated  that  the  committee  played  a  necessary  role  fine-tuning  the 
generic  model  so  it  was  a  consistent,  unified  document  They  felt  as  an  industry  group,  they  had 
developed  a  generic  model  more  suitable  to  industry  than  if  FSIS  had  done  it. 

The  pilot  plants  were  the  first  to  attempt  to  develop  a  plant-specific  plan  from  a  workshop- 
developed  generic  model.  Special  Team  members  became  more  skilled  at  helping  pilot  plants 
develop  plant-specific  plans. 
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Cooked  Sausage  Workshop 

The  Cooked  Sausage  Workshop  was  held  in  Fort  Worth,  Texas,  on  May  21-23,  1991.  This 
workshop  focused  on  cooked  sausage  (frankfurter).  A  total  of  35  participants  and  25  observers 
were  in  attendance.  | 

As  a  result  of  the  evaluation  report  of  the  Refrigerated  Foods  workshop,  the  first  morning  session' 
was  changed.  An  additional  question  was  included  in  the  questionnaire  asking  for  their 
comments  on  that  session.  The  largest  number  of  comments  suggested  shortening  or  eliminating 
the  morning  agenda.  A  slightly  smaller  group  of  respondents  suggested  leaving  the  session  as 
it  was. 

The  steering  committee  was  charged  with  fine-tuning  the  generic  model.  No  unresolved  issues 
were  raised  at  the  workshop.  However,  the  committee  wanted  to  revise  the  model  as  members 
felt  it  contained  too  many  CCPs.  The  FSIS  Special  Team  told  the  committee  they  could  not 
revise  the  model  but  they  could  submit  an  alternative  model,  which  they  did. 

The  generic  model  was  tested  in  three  pilot  plants.  Plant-specific  HACCP  plans  based  on  the 
generic  model  were  successfully  developed  during  the  pilot  tests. 
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Findings 

Sixty-two  percent  of  the  participant  respondents  agreed  that  the  workshop  resulted  in  a  useful 
generic  HACCP  model  developed  by  industry  participants. 

Seventy-six  percent  of  participant  respondents  said  the  workshop  accomplished  what  they 
expected.  Participant  respondents  said  interaction  with  a  good  cross-section  of  peers  and  FSIS 
was  the  most  beneficial  to  them. 

Participant  respondents  said  the  written  materials  were  most  useful  for  "assessing  finish  product 
hazards,"  "assessing  ingredient  hazards,"  and  "developing  assigned  CCP  monitoring  plan." 

Some  participant  respondents  were  concerned  that  discussions  were  impeded  by  what  they  felt 
were  tangential  issues.  However,  observer  respondents  said  the  workshop  was  very  well 
organized. 

Steering  committee  members  said  steering  committees  performed  an  essential  function  by  making 
the  wording  of  the  workshop-developed  generic  models  uniform  to  avoid  misinterpretations  and 
ensuring  consistency.  These  members  expressed  frustration  because  as  workshop  representatives, 
they  thought  they  should  be  able  to  make  substantive  changes  in  the  generic  model. 

Although  the  generic  model  fit  well  in  these  pilots,  some  of  these  plants  took  a  long  time  to 
complete  satisfactory  plant-specific  models.  In  fact,  one  plant  was  still  making  changes  to  the 
model  after  the  implementation  phase  had  begun. 


Poultry  Slaughter  Workshop 


The  Poultry  Slaughter  Workshop  was  held  in  Atlanta,  Georgia,  on  August  27-29,  1991.  This 
was  the  first  workshop  for  a  slaughter  process.  This  workshop  focussed  on  ready  to  cook  whole 
young  chicken.  A  total  of  43  participants  and  54  observers  were  in  attendance. 

In  addition  to  fine-tuning  the  generic  model  developed  at  the  workshop,  the  steering  committee 
was  asked  to  address  "notes  from  industry"  attached  to  the  generic  model  concerning  chemical 
residues  and  chlorination. 

The  generic  model  was  tested  in  three  pilot  plants.  The  plant-specific  models  were  successfully 
developed  using  the  generic  model. 

Findings 

Fifty-six  percent  of  the  participant  respondents  agreed  that  the  workshop  resulted  in  a  useful 
generic  HACCP  model  developed  by  industry  participants. 

Seventy-two  percent  said  that  the  workshop  had  accomplished  what  they  expected.  The  most 
beneficial  aspect  of  the  workshop  to  participants  was  "the  chance  to  discuss  and  generate  various 
options,  meeting  and  talking  with  fellow  members  of  the  workshop"  and  "getting  a  feel  for  the 
general  opinion  of  industry  towards  HACCP." 

Participant  respondents  said  the  written  materials  were  most  useful  in  "determining  placement 
of  CCP's  on  the  flow  diagram." 

Some  participant  respondents  were  disturbed  that  USDA  participants  were  allowed  to  vote  at  the 
workshop.  They  said  this  detracted  from  the  final  package  being  an  industry  model. 

Most  members  of  this  workshop  steering  committee  said  it  was  important  for  industry  to 
participate  on  the  steering  committee  in  order  for  industry  to  claim  ownership  of  the  model. 

One  poultry  slaughter  plant  was  the  first  volunteer  plant  to  have  developed  a  plant-specific  plan 
prior  to  meeting  with  the  Special  Team,  one  plant  had  had  no  prior  experience  with  HACCP,  and 
one  plant  was  initially  uncomfortable  with  the  requirements  of  participating  as  a  volunteer  plant. 


Ground  Beef  Workshop 


The  Ground  Beef  Workshop  was  held  in  Phoenix,  Arizona  on  December  3-5,  1991.  This 
workshop  focused  on  raw  ground  beef.  A  total  of  31  participants  and  12  observers  were  in 
attendance. 

The  steering  committee  was  charged  with  fine-tuning  the  generic  model  developed  at  the 
workshop.  No  unresolved  issues  were  raised  at  the  workshop. 

The  model  developed  at  this  workshop  was  not  tested  in  pilot  plants  following  the  workshop 
because  of  time  constraints  for  the  study. 

Findings 

Eighty-nine  percent  of  the  participant  respondents  agreed  that  the  workshop  resulted  in  a  useful 
generic  HACCP  model  developed  by  industry  participants.  I 

Ninety-three  percent  said  the  workshop  accomplished  what  they  expected.  They  commented  that 
they  had  a  much  better  understanding  of  what  HACCP  is,  how  to  develop  a  specific  plan,  and 
how  to  implement  it  in  the  field. 

Participant  respondents  indicated  that  the  most  useful  written  materials  were  "assessing  finished  I 
product  hazards." 

Members  of  this  steering  committee  stated  they  had  benefitted  from  the  work  of  the  previous 
committees. 
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Swine  Slaughter  Workshop 


The  Swine  Slaughter  Workshop  was  held  in  Minneapolis,  Minnesota,  on  March  31-April  2, 1992. 
This  workshop  focused  on  young  market  hogs.  A  total  of  30  participants  and  six  observers  were 
in  attendance. 

The  National  Advisory  Committee  on  Microbiological  Criteria  for  Foods  expanded  its  original 
report  upon  which  the  HACCP  study  were  based  by  incorporating  in  Principle  #1  a  decision  tree 
intended  to  facilitate  the  identification  of  critical  control  points  (CCPs).  This  workshop  was  the 
first  to  use  the  decision  tree  method  to  identify  CCPs.  A  question  was  added  to  the  questionnaire 
asking  participants  about  it 

In  addition  to  fine-tuning  the  generic  model  developed  at  the  workshop,  the  steering  committee 
was  asked  to  address  as  addenda  to  the  model  issues  of  pesticides  and  antibiotic  residues  and  also 
organic  acid  sanitizing  systems  for  microbial  reduction. 

The  model  developed  at  this  workshop  was  not  tested  in  pilot  plants  following  the  workshop 
because  of  time  constraints  for  the  study. 

Findings 

When  respondents  were  asked  to  comment  on  the  decision  tree  method  used  to  assess  hazards 
and  identify  preventive  measures  and  CCPs,  they  said  the  method  was  useful,  effective  and  would 
aid  the  process  of  preparing  a  HACCP  plan. 

Eighty  percent  of  the  participant  respondents  agreed  that  the  workshop  resulted  in  a  useful 
generic  HACCP  model  developed  by  industry  participants. 

Ninety-six  percent  said  the  workshop  had  accomplished  what  they  expected.  They  said  the 
workshop  gave  them  a  basic  understanding  of  the  principles  and  concepts  necessary  to  formulate 
HACCP  plans  and  set  up  programs  in  their  own  plants.  "It  made  clear  a  viable  method  to  be 
used  in  a  slaughter  situation  to  identify  and  control  hazards." 

The  participant  respondents  said  the  small  group  breakout  sessions  were  the  most  beneficial  part 
of  the  workshop. 

Participant  respondents  indicated  that  the  most  useful  written  materials  were  "assessing  finished 
product  hazards." 

Steering  committee  members  said  their  role  was  to  fine-tune  the  generic  model  for  consistency 
and  eliminate  any  wording  indicating  a  specific  method  of  control  that  must  be  used  at  a  CCP. 
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IH.  CONCLUDING  REMARKS 


It  was  important  to  examine  all  three  components  of  the  workshop  process,  the  workshops 
themselves,  their  steering  committees,  and  the  adaption  process,  to  understand  whether  the 
workshops  were  successful  in  producing  generic  HACCP  models  which  could  be  used  by 
industry. 

According  to  workshop-participants  whojespondedto  the  questionnaire,  useful  generic  HACCP 
models  were  produced  at  the  FSIS/industry  workshops.  They  felt  the  workshop  format  worked 
well  in  developing  HACCP  models.  Eighty-four  percent  of  the  participant  respondents  said  the 
workshop  accomplished  what  they  expected.  When  asked  to  suggest  alternative  methods  for 
developing  generic  HACCP  models,  more  than  half  the  participant  respondents  volunteered  that 
the  workshop  method  was  best  for  developing  generic  models. 

Steering  committee  members  said  steering  committees  were  essential  to  the  success  of  the 
workshop  process.  They  said  steering  committees  played  an  important  role  in  fine-tuning  the 
models  developed  at  the  workshops  for  consistency  both  within  themselves  and  also  with  other 
generic  models.  In  addition,  the  committees  were  able  to  handle  issues  which  were  not  addressed 
completely  at  the  workshops. 

The  generic  models  developed  at  the  workshops  were  found  to  be  applicable  in  volunteer  pilot 
plants.  The  HACCP  Special  Team  requested  only  one  change  in  a  generic  model  to  be  made  by 
a  steering  committee  based  on  plant-specific  plans  developed  in  volunteer  plants. 

However,  some  plants  needed  a  lot  of  help  preparing  their  plant-specific  plan.  Their  ability  to 
prepare  a  plant-specific  plan  was  enhanced  by  prior  knowledge  of  Total  Quality  Control  (TQC), 
Partial  Quality  Control  (PQC),  or  HACCP  type  programs.  More  information  or  training  about 
how  to  design  a  HACCP  plan  or  a  more  detailed  generic  model  would  be  necessary  before  some 
plants  could  develop  a  HACCP  plan  on  their  own. 
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SUMMARY 


In  January  1990,  the  Food  Safety  and  Inspection  Service  (FSIS)  began  a  study  to  determine  the  process 
for  Hazard  Analysis  Critical  Control  Points  (HACCP)*  implementation  in  meat  and  poultry  operations. 
As  part  of  the  study,  five  FSIS/industry  workshops  were  held  to  develop  generic  HACCP  models. 
Workshops  were  held  to  develop  HACCP  models  for  Refrigerated  Foods,  Cooked  Sausage,  Poultry 
Slaughter,  Ground  Beef,  and  Swine  Slaughter. 

Participants  and  observers  at  the  workshops  were  asked  to  complete  a  questionnaire  at  the  completion  of 
the  workshop.  An  examination  of  their  responses  showed  that  the  workshop  format  for  developing 
HACCP  models  worked  well.  Eighty-four  percent  of  the  participants  who  completed  the  questionnaire 
said  the  workshop  accomplished  what  they  expected.  Both  participants  and  observers  said  the  workshop 
format  was  a  sound  approach  for  producing  a  generic  HACCP  model  and  said  that  useful  generic  models 
were  produced  at  the  workshops. 

Workshop  participants  said  they  had  a  better  understanding  of  HACCP  and  FSIS's 
position  on  HACCP  from  attending  the  workshop. 

Workshop  participants  said  the  workshops  were  successful  in  generating  a  generic  model 
for  their  process/product 

Workshop  participants  said  developing  the  generic  model  was  beneficial  to  them  in 
learning  how  to  write  a  HACCP  plan. 

Workshop  participants  said  that  holding  a  workshop  was  the  best  method  to  use  in 
developing  a  generic  HACCP  plan. 

Participants  at  the  Refrigerated  Foods,  Ground  Beef  and  Swine  Slaughter  workshops  were  more  likely  to 
rate  the  workshops  higher  than  participants  at  the  Cooked  Sausage  and  Poultry  Slaughter. 

There  were  no  differences  in  responses  between  participants  at  processing  and  slaughter  workshops.  There 
were  no  differences  in  responses  between  workshop  participants  and  workshop  observers  except  that 
observers  were  more  likely  than  participants  to  feel  the  observer  role  in  the  workshops  was  beneficial. 


♦Hazard  Analysis  Critical  Control  Points  (HACCP)  is  a  process  control  system  that  identifies  "critical"  points  in  the  food  production 
process  that  should  be  controlled  to  prevent  food  safety  hazards  from  occurring. 
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I.  INTRODUCTION 


Purpose 

To  examine  the  role  of  joint  FSIS/industry  workshops  in  the  development  of  generic  HACCP*  models  for 
meat  and  poultry  products. 

Background 

In  January  1990,  the  Food  Safety  and  Inspection  Service  (FSIS)  began  a  study  to  determine  the  process 
for  Hazard  Analysis  Critical  Control  Points  (HACCP)  implementation  in  meat  and  poultry  operations.  As 
part  of  the  study,  five  FSIS/industry  workshops  were  held  to  develop  generic  HACCP  models.  Workshops 
were  held  to  develop  HACCP  models  for  Refrigerated  Foods,  Cooked  Sausage,  Poultry  Slaughter,  Ground 
Beef,  and  Swine  Slaughter.  Participants  in  these  workshops  were  representatives  from  industry,  trade 
associations  and  government.  Workshops  were  open  to  the  public  for  observation,  on  a  reserved  space 
basis.  Time  was  allotted  for  observer  comments,  but  observers  were  not  allowed  to  participate  in  the 
model  development  process. 

The  objectives  of  the  workshops  were  to: 

1.  Assure  participants  understand  the  seven  HACCP  principles  and  their  application. 

2.  Assist  the  industry  participants  in  developing  a  generic  HACCP  model  for  their  process/product. 

3.  Use  a  workshop  design  which  will  help  capture  input  from  all  participants  in  the  industry 
HACCP  model. 

4.  Provide  HACCP  information  needed  for  the  participants  to  tailor  the  generic  HACCP  model 
into  a  specific  HACCP  plan  for  their  products/processes  at  their  establishments. 

These  objectives  were  to  be  met  by  involving  all  participants  in  the  HACCP  model  design  by  structuring 
the  workshop  in  short  group  participation  segments  in  both  large  and  small  group  activities. 

The  workshops  were  facilitated  by  a  Special  Team  of  headquarters  and  field  employees  which  orchestrated 
all  workshop  sessions,  prepared  all  needed  materials,  and  led  the  sessions.  Special  Team  members  were 
FSIS  personnel  selected  for  leadership,  communication  skills  and  a  strong  background  in  FSIS  operations 
and  regulations. 

Five  Subject  Matter  Experts  (SME)  were  chosen  competitively  from  FSIS  field  staff  for  each  workshop. 
Their  role  was  to  provide  technical  advice  on  each  specific  workshop  product/process.  Pre-workshops 
were  held  prior  to  each  workshop  with  the  SMEs  to  work  out  potential  problem  areas  and  plan  the 
workshop  specifics. 


Methodology 

At  the  end  of  each  workshop,  participants  and  observers  were  asked  to  complete  a  questionnaire 
addressing  workshop  design,  materials,  development  of  a  generic  HACCP  model,  the  workshop  approach 
and  any  suggested  improvements. 

♦Hazard  Analysis  Critical  Control  Points  (HACCP)  is  a  process  control  system  that  identifies  "critical"  points  in  the  food  production 
process  that  should  be  controlled  to  prevent  food  safety  hazards  from  occurring. 


These  questionnaire  responses  were  analyzed  and  a  report  written  following  each  workshop.  These  were 
shared  with  the  workshop  staff  and  alterations  were  made  to  the  workshops. 

The  overall  response  rate  was  88  percent  for  participants  varying  from  75  percent  for  the  Refrigerated 
Foods  workshop  to  100  percent  for  the  Swine  Slaughter  workshop.  Overall  response  rate  for  observers 
was  51  percent  varying  from  33  percent  for  the  Swine  Slaughter  workshop  to  72  percent  for  the  Cooked 
Sausage  and  Poultry  Slaughter  workshops. 

Because  of  low  numbers  of  observers  at  some  workshops  and  low  response  rate  of  observers,  findings  will 
be  reported  for  participant  responses  unless  otherwise  noted.  Significant  difference  in  responses  between 
participants  and  observers  will  be  described  where  they  occur. 

The  majority  of  both  participants  and  observers  said  they  were  affiliated  with  the  meat  and  poultry 
industry.  The  next  largest  numbers  were  trade  association  members  for  participants  and  government 
employees  for  the  observer  category. 

Report  Plan 

This  report  will  summarize  the  questionnaire  results  from  five  HACCP  workshops  held  to  develop  generic 
models  for  Hazard  Analysis  and  Critical  Control  Points  for  five  products/processes.  Specific  results  from 
each  of  the  five  workshops  will  be  found  in  the  Appendices.  (See  Appendix  A-E.) 

The  first  section  uses  participant  responses  to  describe  how  well  the  workshops  met  the  objectives  set  by 
the  workshop  organizers.  The  second  section  provides  a  summary  of  all  responses  to  the  workshop 
questionnaires.  The  last  section  addresses  differences  in  responses  between  participants  at  different 
workshops  and  a  summary  of  findings. 
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II.  OBJECTIVES  OF  HACCP  WORKSHOP 


Introduction 

This  section  will  use  participant  responses  to  describe  how  well  the  workshops  met  the  four  objectives  set 
by  workshop  organizers. 

Objective  One:  HACCP  Principles 

The  first  objective  of  the  workshops  was  to  "assure  participants  understand  the  seven  HACCP  principles 
and  their  application."  The  first  day's  sessions  introduced  HACCP  principles  and  the  remaining  workshop 
sessions  were  spent  using  these  principles  to  develop  a  generic  HACCP  model.  Despite  dissatisfaction 
with  the  length  of  the  introductory  materials,  participants  said  they  had  a  better  understanding  of  HACCP 
after  the  workshops  concluded.  Participants  said  that  participating  at  the  workshop  had  given  them 
"insight  into  the  purpose  of  HACCP."  They  had  "learned  what  HACCP  is  and  what  it  may  lead  to  and 
understood  HACCP  and  its  development  and  implementation."  "The  steps  involved  in  the  development 
of  the  HACCP  program  and  the  materials  made  available  for  further  training  when  I  return  to  my  facility" 
were  most  beneficial  to  them. 


Objective  Two:  Model  Development 

The  second  objective  of  the  workshops  was  to  "assist  the  industry  participants  in  developing  a  generic 
HACCP  model  for  their  process/product."  Each  workshop  resulted  in  a  generic  HACCP  model  being 
developed.  Participants  differed  in  their  judgement  of  how  generic  the  model  was.  Some  thought  it  too 
generic  and  would  have  liked  specific  examples.  Others  thought  it  was  good  as  developed. 

Participants  disagreed  on  the  extent  to  which  they  felt  the  developed  model  was  an  FSIS  plan  or  an 
industry  plan.  Some  participants  said  FSIS  had  thought  out  the  generic  plan  ahead  of  time  and  the 
workshop  was  "rubber  stamping"  it.  Others  liked  going  through  the  process  without  any  pre-conditions. 
Some  would  have  liked  having  a  sample  plan  to  follow  from  a  previous  workshop,  others  did  not 

Objective  Three:  Workshop  Design 

The  third  objective  of  the  workshops  was  to  "use  a  workshop  design  which  will  help  capture  input  from 
all  participants  in  the  industry  HACCP  model." 

The  workshop  format  included  a  mix  of  large  group  and  small  group  sessions.  Participants  at  each 
workshop  were  divided  into  five  small  working  groups.  A  subject  matter  specialist  was  assigned  to  each 
small  group.  These  groups  worked  on  individual  critical  control  points  (CCPs)  and  reported  back  to  the 
whole  group.  At  the  whole  group  sessions,  a  computer  run  projection  screen  made  the  deliberations  of 
the  group  visible  to  all.  After  each  session,  copies  of  the  session's  work  were  made  and  distributed  to  all 
participants. 

Eighty-five  percent  of  the  participants  who  responded  rated  the  workshop  as  adequate  or  more  than 
adequate  in  completing  the  workshop  objectives.  Many  participants  stated  their  appreciation  for  the  use 
of  small  breakout  groups.  They  allowed  for  "an  easier  exchange  of  ideas." 
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They  were  also  enthusiastic  about  the  use  of  audiovisuals,  the  computer-run  overhead  projector,  and  the 
fact  that  a  photocopy  machine  was  easily  available.  Observers  stated  that  they  would  have  like  to  have 
had  copies  of  the  participant  manuals.  Both  groups  stated  that  frequently  receiving  hard  copy  updates  of 
workshop  products  was  beneficial. 

Workshop  organizers  were  responsive  to  participant  evaluation  and  made  changes  to  the  workshop  agenda 
in  response  to  their  comments. 

Objective  Four:  Tailor  Generic  Model  to  Plant-Specific  Plan 

The  fourth  objective  was  to  "provide  HACCP  information  needed  for  the  participants  to  tailor  the  generic 
HACCP  model  into  a  specific  HACCP  plan  for  their  products! processes  at  their  establishments." 

All  the  pilot  plants  were  able  to  develop  plant-specific  plans  based  on  the  generic  HACCP  models 
developed  at  the  workshops.  Some  plants  were  more  easily  able  to  develop  a  plant-specific  plan.  This 
depended  upon  how  much  prior  HACCP,  Total  Quality  Control  (TQQ,  or  Partial  Quality  Control  (PQC) 
experience  the  plant  quality  control  staff  had.  Some  plants  would  have  needed  more  training  to  have  been 
able  to  develop  a  plant-specific  plan  on  their  own. 

Some  workshop  participants  commented  that  they  had  learned  enough  information  to  return  to  their 
company  and  develop  a  HACCP  plan  for  their  plant. 
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III.  SUMMARY  FINDINGS  OF  THE  FIVE  HACCP  WORKSHOPS 


Introduction 


At  the  end  of  each  HACCP  workshop,  participants  and  observers  were  asked  to  answer  a  series  of 
questions  about  HACCP  model  development,  workshop  design  and  operation,  and  workshop  materials. 
For  each  of  these  topics,  a  summary  question  was  designed.  Results  of  the  summary  questions  for  each 
topic  are  presented  below. 


HACCP  Model  Development 


Respondents  were  asked  how  successful  the  workshop  was  in  identifying  critical  control  coints  (CCPs), 
determining  critical  limits  for  each  CCP,  developing  monitoring  plan  activity  for  identified  CCPs, 
developing  corrective  action  approaches,  developing  record-keeping  systems,  and  developing  verification 
procedures.  The  following  question  was  asked  as  a  summary  to  these  specific  ones. 

Overall,  how  successful  was  the  workshop  in  developing  a  generic  HACCP  model? 

Very  Successful  Somewhat        Not  successful 

successful  successful         /don't  know 

Participants  14%  55%  26%  5% 

Observers  8%  46%  37%  9% 


Both  participant  and  observer  respondents  stated  the  workshop  was  successful  in  developing  a  generic 
HACCP  model.  However,  participants  tended  to  answer  more  affirmatively. 


Workshop  Design  and  Operation 


Respondents  were  asked  to  rate  the  workshop  in  terms  of  development  of  a  generic  product/process 
description,  development  of  process  flow  diagram,  organization  of  the  three  days,  use  of  small  "break-out 
groups",  reporting  of  small- "break-out"  group  activities  to  entire  group,  opportunity  for  input  from 
observers,  opportunity  for  input  from  participants,  effectiveness  of  facilitators  in  helping  achieve  workshop 
objectives,  effectiveness  of  subject  matter  experts  in  helping  achieve  workshop  objectives,  effectiveness 
of  group  leaders  in  helping  achieve  workshop  objectives,  and  effectiveness  of  group  reporters  in  helping 
achieve  workshop  objectives.  The  following  question  was  asked  as  a  summary  to  these  specific  ones. 

Rate  the  workshop  in  terms  of  completion  of  workshop  objectives. 


More  than  Adequate  Marginally  Inadequate 

adequate  adequate  /don't  know 

Participants           16%  70%  11%  3% 

Observers             14%  50%  23%  13% 
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Both  participant  and  observer  respondents  rated  the  workshop  as  adequate  or  more  than  adequate  in  terms 
of  completing  the  workshop  objectives.  Again,  participants  were  somewhat  more  likely  to  rate  the  design 
and  operation  of  the  workshops  more  positively. 

Workshop  Materials 

Respondents  were  asked  how  useful  the  written  materials  were  for  assessing  finished  product  hazards, 
assessing  ingredient  hazards,  determining  placement  of  CCPs  on  flow  diagram,  determining  critical  limits 
for  each  CCP,  developing  assigned  CCP  monitoring  plan,  developing  corrective  action  approaches, 
developing  record -keeping  systems,  and  developing  verification  procedures.  The  following  question  was 
asked  as  a  summary  to  these  specific  ones. 

Overall,  how  useful  were  the  written  materials  for  developing  a  generic  HACCP 
model? 

Very  Useful  Useful  Somewhat       Not  at  all  Useful 

Useful 

Participants  16%  61%  19%  4% 

Observers        Observers  were  asked  not  to  complete  this  section  since  they  did  not  have  the  complete 
set  of  workshop  materials. 

Seventy-seven  percent  of  the  participant  respondents  indicated  that  the  written  materials  were  useful  or 
very  useful. 

Workshop  Expectations 

Respondents  were  asked  if  the  workshops  had  accomplished  what  they  expected.  Overall,  84  percent  of 
the  participant  respondents  thought  the  workshop  accomplished  what  they  had  expected. 


DID  THE  WORKSHOP  ACCOMPLISH  WHAT  YOU  EXPECTED? 


100 
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Respondents  were  given  the  opportunity  to  provide  comments.  One  hundred  three  participant  respondents 
wrote  comments  which  are  summarized  below. 

Sixty-nine  respondents  said  the  workshop  accomplished  what  they  had  expected.  Of 
these,  forty-one  said  an  acceptable  generic  model  for  the  product/process  in  their 
workshop  was  developed  that  could  be  adapted  by  the  industry.  Seventeen  felt  they  had 
a  better  understanding  of  HACCP  and  FSIS'  position  about  HACCP  from  attending  the 
workshop.  Seven  remarked  that  there  were  excellent  information  exchanges,  good 
discussion,  consensus  and  good  final  decisions  at  the  workshop.  Four  others  made  general 
laudatory  remarks. 

Eight  answered  both  yes  and  no.  They  said  the  definition  of  "generic"  was  unclear  as  was 
the  reason  for  developing  a  generic  plan. 

Twenty-six  said  the  workshop  did  not  accomplish  what  they  expected.  Of  these,  nineteen 
said  the  model  developed  was  too  generic,  the  CCPs  were  watered  down,  or  had  too  many 
CCPs.  Seven  responses  expressed  general  dissatisfaction  with  the  workshop.  They  said 
they  expected  to  gain  more  from  the  workshop. 

Respondents  were  asked  if  the  individual  sessions  accomplished  what  they  expected.  Only  one  person 
said  that  none  of  the  sessions  accomplished  what  he  expected.  The  most  frequent  answer  for  the 
Refrigerated  Foods,  Cooked  Sausage,  Ground  Beef  and  Swine  Slaughter  workshops  was  "Yes  for  all." 
For  Poultry  Slaughter,  the  most  common  response  was  "Yes  for  some." 

Many  of  the  respondents  assumed  that  by  "individual  sessions"  the  questionnaire  was  referring  to  the  small 
group  meetings  only,  rather  than  all  the  sessions.  Respondents  were  given  the  opportunity  to  provide 
comments.  Seventy-three  participant  respondents  wrote  comments  which  are  summarized  below. 

Thirty-one  respondents  said  it  was  advantageous  to  work  in  smaller  groups  which 
facilitated  more  in-depth  discussion  with  full  participation  of  all  participants. 

Fourteen  commented  on  the  allocation  of  time  at  the  workshops.  Some  said  the  amount 
of  time  allotted  to  small  group  meetings  was  too  short,  some  felt  too  much  time  was  spent 
on  some  issues,  and  some  felt  that  some  sessions  were  a  waste  of  time. 

Seven  said  the  workshop  format  made  good  use  of  different  ways  of  developing  a 
HACCP  plan,  for  example,  small  groups,  large  group,  facilitator,  group  leaders,  recorders, 
etc. 

Seven  said  they  learned  a  lot  about  HACCP  and  writing  a  HACCP  plan,  seven  said  the 
process  was  well  organized  and  took  into  account  different  size  firms,  and  seven  made 
other  comments. 


Respondents  were  asked  to  suggest  methods  other  than  workshops  [that]  could  be  used  to  develop 
HACCP  models?  One  hundred  twenty  participant  respondents  wrote  comments  which  are  summarized 
below. 
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Sixty-nine  respondents  said  the  workshop  format  was  the  best  one  to  use  to  develop 
HACCP  generic  models.  "This  is  the  best  method  with  industry  being  able  to  actively 
participate  in  the  development  of  future  FSIS  regulations."  "This  is  probably  the  best 
method  for  all  to  air  their  options."  "The  open  forum  was  useful  and  educational  for 
many  of  the  participants."  "The  workshop  was  the  most  efficient  method  for  reaching 
agreement  among  the  various  parties."  "Cooperation  of  industry  and  USDA  is  the 
preferable  format." 

The  question  posed  three  alternative  suggestions  to  the  workshop  for  HACCP  model 
development.  From  these,  21  respondents  said  that  alternative  C:  Submission  to  FSIS  of 
independently-generated  models  was  best.  Seven  said  that  alternative  A:  FSIS-generated 
model  published  in  the  Federal  Register  was  best.  Four  said  alternative  B:  Development 
of  a  model  by  selected  academicians  was  best.  Two  suggested  a  combination  of  two  of 
the  options. 

Seventeen  made  other  comments  including  these  suggestions: 

•  Delphi  technique  to  solicit  beginnings  of  a  generic  model  with  finalization  at  actual  meeting. 

•  Input  from  trade  associations. 

•  Models  generated  by  industry  members  who  have  been  applying  HACCP  for  at  least  a  year. 

•  Submission  of  independently-generated  models  as  resource  material  for  a  panel  of  selected 
academicians  and  qualified  FSIS  employees  to  generate  a  model  plant  which  would  be 
published  in  the  Federal  Register  for  comments. 

•  FSIS/academician  model  submitted  as  example  for  workshop. 

•  Run  a  pilot  in  one  or  two  plants  and  circulate  the  proposed  developed  model  to  the  rest  of  the 
industry  for  comments. 

•  Small  group  (less  than  20  people)  to  review  HACCP  programs  independently  generated  that 
have  already  been  implemented  successfully. 

•  Have  a  steering  committee  develop  a  model  then  present  it  at  a  workshop  for  completion. 

Respondents  were  asked  what  was  most  beneficial  to  you  about  the  workshop?  One  hundred  forty-six 
participant  respondents  wrote  comments  which  are  summarized  below. 

Sixty-eight  respondents  said  "The  open  forum  between  government  and  industry  [was 
most  beneficial]  because  it  allowed  everyone  to  add  their  comments  and  suggestions  and 
allowed  for  sharing  of  many  different  points  of  view." 

Thirty-eight  said  training  in  HACCP  and  working  on  the  model  were  most  beneficial. 
Twenty-eight  said  that  the  small  groups  were  the  best  part  of  the  workshop. 
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Seven  cited  other  factors  including  consensus,  politics,  and  everything,  and  five  cited  the 
computer  system,  overheads,  etc.  were  most  beneficial. 

Respondents  were  asked  what  was  least  beneficial  to  you  about  the  workshop?  One  hundred  five 
participant  respondents  made  comments  which  are  summarized  below. 

Twenty-six  respondents  felt  the  opening  session  and  review  of  HACCP  theory,  and/or  risk 
assessment  sessions  took  too  long. 

Twenty-four  felt  too  much  time  was  wasted.  Of  these,  thirteen  commented  that  too  much 
time  was  spent  arguing  over  trivia.  Eleven  felt  the  length  of  the  workshop  could  have 
been  shortened. 

Twenty-two  said  the  workshop  facilities,  the  workshop  facilitators,  or  workshop 
procedures,  for  example,  the  observers  and  voting,  was  least  beneficial  to  them. 

Fifteen  commented  that  everything  was  beneficial. 

Six  felt  the  role  of  the  Subject  Matter  Experts  was  unclear  and  five  were  discontented 
with  some  part  of  the  HACCP  model  development  process. 

Seven  made  other  comments. 


Respondents  from  three  workshops  were  asked  to  evaluate  the  first  morning  agenda.  When  respondents 
at  the  Refrigerated  Foods  workshop  said  the  first  morning  session  was  too  long,  the  workshop  organizers 
made  alterations  in  the  session.  A  question  was  added  to  the  questionnaire  for  respondents  at  the  Cooked 
Sausage,  Poultry  Slaughter,  and  Ground  Beef  Workshops  asking  them  specifically  to  evaluate  the  first 
morning  agenda.  Seventy-three  participant  respondents  made  comments  which  are  summarized  below. 

Forty  respondents  suggested  shortening,  modifying  or  eliminating  the  first  morning 
session.  "Too  much  time  was  spent  on  introductions  and  mechanics  and  the  HACCP 
overview  should  be  shortened." 

Twenty-three  said  the  first  morning  agenda  was  fine  as  is  and  there  was  no  need  to 
change  it. 

Five  suggested  sending  an  information  packet  to  participants  prior  to  the  workshop  and 
asking  them  to  familiarize  themselves  with  the  materials  before  attending  the  workshop. 

Five  would  have  liked  more  of  a  review  of  HACCP  during  the  first  session. 


The  Swine  Slaughter  workshop  was  the  only  workshop  held  after  the  National  Academy  of  Sciences 
Committee  on  Microbiological  Criteria  for  Food  (NASCMF)  changed  HACCP  Principle  #1.  Respondents 
were  asked  comment  on  Principle  #1  and  the  use  of  a  decision  tree  to  assess  hazards  and  identify 
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preventive  measures  and  CCPs.  Twenty-eight  participant  respondents  made  comments  which  are 
summarized  below. 

Twenty-six  respondents  said  the  decision  tree  was  a  very  good  method  to  identify  and 
assess  hazards.  It  was  simple  but  effective  in  determining  critical  control  points. 

Two  said  the  method  was  adequate  and  acceptable. 

At  the  end  of  the  questionnaire,  respondents  were  given  the  opportunity  to  provide  additional  comments 
about  the  workshop.  One  hundred  twenty-four  participant  respondents  made  comments  which  are 
summarized  below. 

Thirty-nine  respondents  made  comments  or  suggestions  about  how  future  workshops  could 
be  run.  These  suggestions  ran  the  gamut  from  room  size  to  color-coded  name  tags,  to 
including  "float  time"  Friday  mornings  to  use  if  necessary. 

Thirty-three  said  overall  the  workshops  were  excellent.  The  presentations,  visual  aids,  and 
USDA  planning  were  excellent.  "The  workshop  was  exceptional  and  productive." 

Fifteen  voiced  some  confusion  about  the  role  of  USDA,  the  role  of  industry,  and  the  role 
of  the  HACCP  models  developed  at  the  workshops. 

Twelve  felt  the  first  part  of  the  workshop  could  have  been  condensed  to  save  time,  that 
the  workshop  started  off  too  slowly. 

Eleven  made  comments  about  HACCP  and  how  it  related  to  TQC  or  other  current 
industry  or  FSIS  programs. 

Ten  expressed  mixed  feelings  about  the  availability  of  output  from  previous  workshops. 
While  some  felt  previously  generated  generic  models  were  very  useful  to  their  workshop, 
others  disagreed  and  said  previously  generated  generic  models  were  unrelated  to  their 
product 

Four  made  suggestions  for  follow-up  to  the  workshop,  one  felt  the  workshop  was 
somewhat  confusing  at  times  and  one  said  there  was  excessive  FSIS  involvement. 
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IV.  DIFFERENCE  BETWEEN  WORKSHOPS 


Introduction 

The  previous  section  provided  summary  information  about  all  participants  in  all  workshops.  This  section 
will  examine  the  data  from  the  individual  workshops  to  see  if  there  were  any  differences  in  how 
participants  answered  the  questionnaire  based  on  the  workshops  they  attended.  Observer  differences  are 
not  addressed.  There  were  few  differences  between  participants  and  observer  responses.  The  largest 
difference  was  their  response  as  to  whether  observers  were  beneficial  to  the  workshop.  Observers  were 
more  likely  to  agree  that  observers  were  beneficial  to  the  workshop. 

In  addition  to  being  asked  the  series  of  questions  on  HACCP  model  development,  workshop  design  and 
operation,  and  workshop  materials,  participants  were  asked  to  indicate  their  degree  of  agreement  with  four 
statements  concerning  the  workshops. 


HACCP  Model  as  a  Viable  Framework 


The  first  statement  asked  if  respondents  agreed  that  "the  HACCP  model  developed  at  the  workshop  is  a 
viable  framework  for  the  industry  to  implement  on  a  test  basis." 


HACCP  model  developed  at  workshop 
is  a  viable  framework  to  implement  on  test  basis 


100 


H  Ref rig.  Foods 
11  Cooked  Sausage 
H  Poultry  Slaughter 
■  Ground  Beef 
Ll  Swine  Slaughter 


Disagree/neutral 


participants 
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Although  a  large  percentage  of  workshop  participants  agree  with  this  statement,  there  was  some  variation 
among  the  workshops.  Participants  at  the  Poultry  Slaughter  workshop  were  least  likely  to  agree  that  the 
HACCP  model  developed  was  a  viable  framework  to  implement  on  a  test  basis.  The  Swine  Slaughter 
participants  were  most  likely  to  agree  with  the  statement. 


Statement  Two:  Workshop  Format  as  a  Sound  Approach 

The  second  statement  asked  if  respondents  agreed  that  "the  workshop  format  offered  a  sound  approach 
for  producing  a  generic  HACCP  model  for  other  meat  and  poultry  products." 


Workshop  format  offers  a  sound  approach 
for  producing  a  generic  HACCP  model 


100 


Agree 
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H  Refrig.  Foods 
H  Cooked  Sausage 
11  Poultry  Slaughter 
H  Ground  Beef 
I  Swine  Slaughter 


Disagree/Neutral 


participants 


When  the  answers  to  this  question  were  examined  by  workshop,  a  strong  pattern  of  participant  support 
appeared  across  all  workshops.  There  was  very  little  variation  across  workshops. 
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Presence  of  Observers 


The  third  question  asked  if  respondents  agreed  that  "the  presence  of  observers  was  beneficial  to  their 
workshop."  Results  from  both  participants  and  observers  are  reported  here  because  of  response 
differences.  Observers  were  much  more  likely  to  agree  that  their  presence  was  beneficial  to  the  workshop. 
This  difference  makes  intuitive  sense  especially  in  light  of  comments  from  some  observers  who  would 
have  liked  to  have  had  a  larger  role  in  the  workshops. 


Presence  of  observers  was 
beneficial  to  the  workshop. 


Agree  Disagree/neutral 

Participant  Responses 
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Presence  of  observers  was 
beneficial  to  the  workshop. 
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Note  that  for  the  observer  graph,  the  Ground  Beef  workshop  bar  represents  only  six  observers  and  the 
Swine  Slaughter  bar  represents  only  two  observers. 
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Workshop  Resulted  in  Useful  Generic  Model 


The  fourth  statement  asked  if  respondents  agree  that  "the  workshop  resulted  in  a  useful  generic  HACCP 
model  developed  by  industry  participants. 


The  workshop  resulted  in  a  useful  generic  model 
developed  by  industry  participants. 
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The  majority  of  respondents  agree  that  their  workshops  resulted  in  a  useful  generic  model.  Participants 
at  the  Refrigerated  Foods  and  Ground  Beef  workshops  were  most  likely  to  respond  positively. 


Summary  Findings 

There  did  not  appear  to  be  differences  in  the  way  processing  workshop  respondents  rated  the  workshops 
as  compared  to  slaughter  workshop  respondents.  From  the  previous  graphs,  it  can  be  seen  that  Cooked 
Sausage  and  Poultry  Slaughter  workshop  participants  appeared  to  be  the  most  critical  of  the  workshop 
process.  In  like  manner,  Refrigerated  Foods,  Ground  Beef  and  Swine  Slaughter  workshop  participants  were 
most  favorable  to  the  workshops.  Overall,  all  workshop  participants  were  generally  favorable  to  the 
workshop  process. 

According  to  the  responses  of  workshop  respondents,  the  objectives  stated  by  workshop  organizers  were 
met. 

1.  Workshop  participants  said  in  their  ratings  and  in  their  comments  that  they  had  a  better 
understanding  of  HACCP  and  FSIS's  position  on  HACCP  after  attending  the  workshop. 
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2.  Workshop  participants  said  the  workshops  were  successful  in  generating  a  generic  model  for 
their  process/product. 

3.  Workshop  participants  said  the  workshop  format,  especially  the  small  working  groups, 
facilitated  more  in-depth  discussion  with  full  participation  of  all. 

4.  Workshop  participants  said  developing  the  generic  model  was  beneficial  to  them  in  learning 
how  to  write  a  HACCP  plan. 

Workshop  participants  reiterated  that  holding  a  joint  FSIS/industry  workshop  was  the  best  method  to  use 
in  developing  a  generic  HACCP  plan. 
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Summary  of  the 
Evaluation  Results  of  the 
Workshop  for  HACCP 
Model  Development 


Baltimore,  Maryland,  February  26  -  28,  1991 


I 


INTRODUCTION 


This  report  presents  the  evaluation  results  from  the  Food  Safety  Inspection  Service  (FSIS) 
of  the  United  States  Department  of  Agriculture  (USD A)  workshop  to  develop  a  generic 
model  for  a  hazard  analysis  and  critical  point  (HACCP)  plan  for  refrigerated  foods. 
Representatives  from  industry,  trade  associations  and  government  met  in  Baltimore, 
Maryland,  February  26  -  28, 1991,  for  this  workshop  which  focussed  on  "keep  refrigerated" 
products  containing  un cured  meat  or  poultry  that  are  cooked,  assembled,  then  packaged. 
The  workshop  was  open  to  the  public  for  observation,  on  a  reserved  space  basis. 

Participants  and  Observers  were  asked  to  complete  a  questionnaire  which  asked  them 
questions  related  to  the  design  and  operation  of  the  workshop,  the  workshop  materials,  the 
process  employed  in  developing  a  generic  HACCP  model,  and  their  views  concerning  the 
workshop  approach  and  any  suggested  improvements.  Of  the  40  Participants  and  42  Observers 
in  attendance,  30  Participants  (75%)  and  16  Observers  (38%)  responded  to  the  questionnaire. 

This  evaluation  activity  is  one  in  a  series  of  on-going  evaluation  tasks  that  are  a  part  of  the 
HACCP  Implementation  Study.  The  results  of  this  evaluation  should  serve  to  help  plan 
subsequent  workshops  and  to  provide  background  information  to  aid  in  future  planning 
decisions  related  to  HACCP. 


REPORT  PLAN 


This  report  is  organized  into  two  sections.  Section  I  presents  the  results  and  summary  of 
Participants'  responses  and  Section  II  presents  those  for  the  Observers.  Results  of  the 
multiple-choice  questions  are  presented  in  a  format  based  on  that  of  the  original  questionnaire. 
The  responses  are  reported  by  number  and  percent  of  response  for  each  category.  Percentages 
reported  are  based  on  the  number  of  responses  to  each  question  and  do  not  always  total 
100  percent  due  to  arithmetic  rounding. 

Results  of  the  open-ended  questions  are  also  summarized  in  each  Section.  A  content  analysis 
was  performed  for  all  hand-written  comments.  Respondents'  remarks  are  paraphrased 
and  grouped  by  question.  Some  respondents  made  multiple  comments  while  others  made 
none,  thus  the  number  of  comments  per  question  does  not  equal  the  number  of  respondents. 
The  number  of  respondents  who  concurred  with  each  comment  is  listed  parenthetically. 


SECTION  I—  PARTICIPANTS 


SUMMARY  OF  PARTICIPANTS'  RESPONSES  TO 
EVALUATION  QUESTIONNAIRE 


Refrigerated  Foods,  February  26  -  28, 1991 


SUMMARY  OF  PARTICIPANTS'  RESPONSES 


RESPONDENT  CHARACTERISTICS 


1.  Which  grouping  below  best  characterizes  your  current  affiliation? 

Meat  &  Poultry  Industry:    24/80%  Consultant  to  Meat  &  Poultry  Industry:  1/3% 

Trade  Association:  4/13%  Salad  Manufacturing:  1/3% 

2.  If  you  marked  "Meat  &  Poultry  Industry"  in  Question  1,  please  indicate  all  groupings  below  which  best 
characterize  your  industry  type  and  the  total  production  volume  of  your  establishments).  [Low  Volume 
Establishment  =  1  -  59,999  lbs.  produced /Quarter;  Medium  Volume  Establishment  =  60,000  - 1,000,000  lbs. 
produced /Quarter;  High  Volume  Establishment  =  more  than  1,000,001  lbs.  produced /Quarter] 


Meat  &  Poultry  Industry 


Poultry  Industry 

3.  Days  attended  (mark  all  that  apply): 

4.  Please  check  one: 

HACCP  MODEL  DEVELOPMENT 


Low  Volume  Establishments):  1/4% 

Medium  Volume  Establishments):  7/29% 

High  Volume  Establishments):  10/42% 

Volume  Unspecified:  2/8% 

Low  Volume  Establishments):  1/4% 

Medium  Volume  Establishments):  0/0% 

High  Volume  Establishments):  3/13% 


IDay:  1/3% 
Observer:  0 


2  Days:  1/3% 
Participant  30 


3  Days:  28/93% 


Very  Somewhat  Not  Very  Don't 

Successful  Successful  Successful  Successful  Unsuccessful  Know 


5.    Overall,  how  successful  was  the  workshop  in: 

a.  Identifying  Critical  Control  Points  (CCPs)?  2/7% 

b.  tetenrtining  critical  limits  for  each  CCP?  3/10% 

c.  Developing  monitoring  plan  activity  for  4/13% 
identified  CCPs? 

d.  Developing  corrective  action  approaches?  2/7% 

e.  Developing  record-keeping  systems?  2/7% 

f .  Developing  verification  procedures?  4/14% 

g.  Developing  a  generic  HACCP  model?  4/14% 


22/73% 

6/20% 

0/0% 

0/0% 

0/0% 

17/57% 

10/33% 

0/0% 

0/0% 

0/0% 

22/73% 

4/13% 

0/0% 

0/0% 

0/0% 

23/79% 

4/14% 

0/0% 

0/0% 

0/0% 

23/79% 

3/10% 

0/0% 

1/3% 

0/0% 

19/68% 

4/14% 

0/0% 

1/4% 

0/0% 

17/61% 

6/21% 

0/0% 

0/0% 

1/4% 

NOTE:  Results  are  presented  as  number  of  responses/%  of  responses  for  each  category. 
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DESIGN  AND  OPERATION  OF  WORKSHOP 

6. 

Please  rate  the  workshop  in  terms  of: 

Mart  than 
Adequate 

Adequate 

Marginally 
Adequate 

Inadequate 

Vrry 

Inadequate 

Don't 
Know 

a.  Development  of  a  generic  product/process  5/17% 
description 

22/76% 

2/7% 

0/0% 

0/0% 

0/0% 

b.  Development  of  process  flow  diagram 

4/13% 

23/77% 

3/10% 

0/0% 

0/0% 

0/0% 

c.  Organization  of  the  three  days 

8/27% 

21/70% 

1/3% 

0/0% 

0/0% 

0/0% 

d.  Use  of  small  "break-out"  groups 

13/45% 

15/52% 

1/3% 

0/0% 

0/0% 

0/0% 

e.  Reporting  of  small  "break-out"  group 
activities  to  entire  group 

9/30% 

20/67% 

1/3% 

0/0% 

0/0% 

0/0% 

f .  Opportunity  for  input  from  Observers 

6/21% 

11/38% 

5/17% 

5/17% 

1/3% 

1/3% 

g.  Opportunity  for  mput  from  Participants 

17/57% 

13/43% 

0/0% 

0/0% 

0/0% 

0/0% 

h.  Effectiveness  of  facilitators  in  helping 
achieve  workshop  objectives 

6/20% 

22/73% 

2/7% 

0/0% 

0/0% 

0/0% 

i.  Effectiveness  of  subject  matter  experts  in 
helping  achieve  workshop  objectives 

5/17% 

19/63% 

5/17% 

1/3% 

0/0% 

0/0% 

j.  Effectiveness  of  group  leaders  in  helping 
achieve  workshop  objectives 

6/20% 

24/80% 

0/0% 

0/0% 

0/0% 

0/0% 

k.  Effectiveness  of  group  reporters  in  helping  4/14% 
achieve  workshop  objectives 

24/86% 

0/0% 

0/0% 

0/0% 

0/0% 

1.  Completion  of  workshop  objectives. 

4/14% 

24/83% 

0/0% 

0/0% 

0/0% 

1/3% 

Strongly 
Agree 

Agree 

Neutral 

Disagree 

Strongly 
Disagree 

Don't 
Know 

7. 

Please  indicate  your  degree  of  agreement 
with  each  of  the  following  statements. 

a.  The  HACCP  model  developed  by  the 
workshop  participants  is  a  viable  frame- 
work for  the  refrigerated  food  industry 
to  implement  on  a  test  basis. 

2/7% 

21/70% 

5/17% 

1/3% 

0/0% 

1/3% 

b.  The  workshop  format  offers  a  sound 
approach  for  producing  a  generic  HACCP 
model  for  other  meat  and  poultry  products. 

4/14% 

21/72% 

2/7% 

2/7% 

0/0% 

0/0% 

c.  The  presence  of  Observers  was  beneficial 
to  the  workshop. 

3/10% 

13/43% 

12/40% 

1/3% 

1/3% 

0/0% 

d.  The  workshop  resulted  in  a  useful  generic 
HACCP  model  developed  by  industry 
participants. 

5/17% 

23/77% 

2/7% 

0/0% 

0/0% 

0/0% 
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WORKSHOP  MATERIALS  (Observers:  Please  skip  Question  8  and  continue  with  Question  9.) 

Useful 


Very 
Useful 


Somewhat 
Useful 


Not 
Useful 


Not  At  All 
Useful 


8.     Overall,  how  useful  were  the  written  materials  for: 

a.  Assessing  finished  product  hazards? 

b.  Assessing  ingredient  hazards? 

c.  Determining  placement  of  CCPs  on  flow 
diagram? 

d.  Determining  critical  limits  for  each  CCP? 

e.  Developing  assigned  CCP  monitoring  plan? 

f .  Developing  corrective  action  approaches? 

g.  Developing  record-keeping  systems? 

h.  Developing  verification  procedures? 

i.  Developing  a  generic  HACCP  model? 

WORKSHOP  EXPECTATIONS 


2/7% 

16/55% 

9/31% 

2/7% 

0/0% 

1/4% 

9/32% 

8/29% 

8/29% 

2/7% 

3/11% 

16/59% 

8/30% 

0/0% 

0/0% 

1/3% 

16/55% 

11/38% 

1/3% 

0/0% 

1/3% 

20/69% 

8/28% 

0/0% 

0/0% 

3/10% 

18/62% 

7/24% 

1/3% 

0/0% 

2/7% 

17/59% 

9/31% 

1/3% 

0/0% 

1/4% 

18/64% 

7/25% 

2/7% 

0/0% 

1/3% 

18/62% 

9/31% 

1/3% 

0/0% 

Yes: 

23/92% 

No: 

2/8% 

9.    Did  the  workshop  accomplish  what  you  expected? 
Please  explain. 

(21  comments) 

(6)   Yes.  An  acceptable  generic  model  for  cooked /assembled  chilled  foods  was  developed.  The  model 
should  be  helpful  in  proceeding  forward  with  the  study. 

(5)   Yes.  In  fact  exceeded  my  expectations.  Soliciting  industry  input  is  a  positive  step  in  this  process.  I  feel 
we  had  an  impressive  group  of  the  best  industry  experts  who  "hammered  our"  a  fairly  universally 
usable,  generic  model  HACCP  plan.  Good  group  input.  Interaction  with  others  (Participants  & 
Observers)  was  excellent. 

(5)   Both  Yes  and  No.  As  Participants,  we  seemed  to  have  some  trouble  agreeing  on  the  definition  of  the 
term  generic  and  the  generic  program  developed  was  somewhat  more  generic  than  expected.  Need 
several  examples  to  address  concerns  which  are  applicable  to  all  variations  of  product  categories.  It 
might  have  been  of  benefit  to  have  had  some  way  of  recording  more  specific  issues  which  were 
discussed.  Perhaps  it  may  have  been  better  to  develop  2  models:  1)  a  generic  model  as  we  have 
done,  plus  2)  a  specific  model  to  be  used  as  an  example. 

(3)  Both  Yes  and  No.  The  reason  for  developing  a  generic  plan  was  unclear.  There  seemed  to  be  two 
main  objectives —  that  of  having  industry  Participants  give  their  stamp  of  approval  on  a  plan 
and  that  of  creating  a  plan  that  can  serve  as  a  basis  for  implementation  in  individual  plants. 
These  two  objectives  were  in  conflict  at  times  because  we  were  reluctant  to  commit  to  specificity 
in  the  model  because  of  an  underlying  belief  that  this  model  will  be  used  by  FSIS  as  the  basis  for 
a  regulation. 

(2)   Yes.  We  currently  don't  have  a  HACCP  Program  so  it  was  very  informative  on  how  to  develop  a 
program.  Gave  me  an  opportunity  to  better  understand  HACCP. 
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10.    Did  the  individual  sessions  accomplish  what  you      Yes  for 
expected?  Please  explain.  All: 


Yes  for 
Some:  11/41% 


No  for 
All:  0/0% 


1 


16/59% 


(17  comments) 


(7)    Yes.  It  was  advantageous  to  work  in  smaller  groups  which  facilitated  a  more  in-depth  discussion  with 
full  participation  of  all  Participants.  This  was  a  valuable  time  for  the  shy  people  to  express  their  views 
and  concerns.  Sensitivity  to  these  opinions  is  very  important  in  developing  a  meaningful  model.  I  was 
able  to  contribute  more  to  these  sessions  and  to  grasp  one  idea  before  going  on  to  another  because  in 
each  session  the  program  was  further  developed  until  it  was  completed. 

(2)   Yes.  Excellent  discussion  medium,  good  interaction  and  good  plan.  Our  break  out  facilitator  did  an 
excellent  job  in  keeping  the  group  on  track.  All  sessions  were  conducted  with  objectivity. 

(2)   Yes  for  Some.  I  wish  that  more  time /sessions  would  have  been  allocated  for  small  group  sessions. 
There  should  be  more  time  allotted  for  discussion  and  debate.  USDA  must  realize  that  our  views  of 
HACCP  implementation  will  (and  should)  evolve. 

(2)   Yes  for  Some.  Specific  information  was  discussed  in  the  individual  sessions  that  was  not  conveyed  to 
the  final  program.  I  had  hoped  more  specific  statements  would  have  been  developed,  including 


(2)   Yes  for  Some.  Perhaps  revamp  placement  of  individual  Participants  into  small  groups.  Have  more 
integrated  personalities  and  positions.  Need  to  include  producers  of  referenced  product  in  each 
break-out  group. 

(2)   Yes  for  Some.  Too  many  agendas  were  being  considered  as  people  applied  specifics  to  other 
areas  of  future  concern.  Some  of  our  sessions  got  hung  up  on  "hair-splitting." 


11.  Methods  other  than  a  workshop  format  could  have  been  employed  to  develop  a  generic  HACCP  model. 
Examples  of  options  include  an  FSIS  request  for  (a)  comments  on  an  FSIS-generated  model  published  in 
the  Federal  Register;  (b)  development  of  a  model  by  selected  academicians;  or  (c)  submission  to  FSIS  of 
independently-generated  models.  Please  indicate  your  preference  for  one  of  the  above-listed  options, 
or  describe  any  others  not  mentioned  that  you  would  like  to  be  considered  by  FSIS  in  the  future. 

(20  comments) 

(9)  The  workshop  format  was  very  good.  This  was  the  most  efficient  and  politically  suitable.  To 
improve  it,  perhaps  some  preliminary  work  could  have  been  done.  I  liked  what  "we"  did! 

(3)  Prefer  c  as  these  would  reflect  actions  by  the  industry  who  must  apply  the  HACCP  System.  Suggest 
generated  models  by  industry  members  who  have  been  applying  this  for  at  least  1  year.  This  may 
provide  input  that  was  not  generated  in  the  workshop. 

(2)  The  workshop  and  c  could  be  combined,  for  example,  independently-generated  models  could  have 
been  submitted  for  evaluation  at  the  workshop. 

(1)  It  was  preferable  to  develop  a  HACCP  program  from  scratch  because  the  process  illustrated  some  of 
the  shortcomings  of  the  process.  The  experience  of  this  workshop  along  with  the  experiences  of  future 
workshops  should  be  useful  in  fine-tuning  the  HACCP  process. 

(1 )  A  and  b  should  both  be  pursued  prior  to  any  implementation. 


examples. 


COMMENTS 
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(1 )    A  or  b  are  not  workable  at  all  and  c  might  work,  but  many  companies  would  probably  never  get 
around  to  responding.  Perhaps  input  by  trade  associations  is  another  viable  alternative. 

(1)    Use  b:  development  of  a  model  by  selected  academicians. 

(1)    Maybe  using  the  Delphi  technique  to  solicit  beginnings  of  a  generic  model  with  finalization  at  actual 
meeting. 

(1)   Perhaps  use  submission  of  independently-generated  models  as  resource  material  for  a  panel  of  selected 
academicians  and  qualified  FSIS  employees  to  generate  a  model  plan  which  would  be  published  in  the 
Federal  Register  for  comments. 


12.  What  was  most  beneficial  to  you  about  the  workshop? 
(26  comments) 

(16)  Group  discussion  and  debate  with  peers.  The  open  forum  between  government  and  industry  because 
it  allowed  everyone  to  add  their  comments  and  suggestions  and  allowed  for  sharing  of  many  different 
points  of  view. 

(2)   Consensus.  The  group  think  process  showed  once  again  that  "two  heads  are  better  than  one."  The 
group  developed  a  system  that  was  better  than  the  optional  methods  listed  in  Question  11. 

(2)   Training  in  HACCP  and  an  explanation  of  the  steps  of  the  generic  model,  steering  committee  and 
testing  plants. 

(2)   A  detailed  explanation  of  USDA's  current  thoughts  of  HACCP,  and  a  good  opportunity  to  assist  in 
the  development  of  the  USDA's  HACCP  plan.  I  perceived  a  "refreshing  openness"  exhibited  by  USD  A. 
You  should  be  commended  for  this  approach. 

(1)   Recognizing  that  the  audience  has  been  involved  with  HACCP  in  a  practical  sense  and  that  we  may  be 
further  along  in  the  HACCP  process  than  we  thought. 

(1 )   Open  discussions  in  the  small  group. 

(1 )  The  use  of  the  overhead  projections  was  great. 

(1 )   I  found  the  entire  three-day  workshop  to  be  most  beneficial. 


13.  What  was  least  beneficial  to  you  about  the  workshop? 
(22  comments)  — 

(3)   Hazard  risk  assessment:  the  detailed  discussion  of  Principle  1  took  too  much  time  to  defend  the 
existing  system  rather  than  explain  the  usefulness  of  the  assigned  risk  numbers. 

(3)   The  observer  comments. 

(3)  Nothing  had  no  value. 

(2)  Some  people  and  groups  tended  to  have  some  very  strong  positions  that  were  not  necessarily  the 
wish  of  all  concerned,  but  tended  to  end  up  as  "approved  actions."  Some  parties  had  an  agenda 
independent  of  cooked  and  assembled  chilled  foods. 
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(2)    Some  points  were  discussed  too  long  and  seemed  to  get  off-track.  Many  concepts  were  reviewed 
several  times.  While  I  think  participation  in  activities  such  as  this  is  valuable,  I  would  rarely  have 
3  days  to  devote  to  them. 

(2)  First  a.m.  session.  The  introductions,  even  though  they  may  have  been  necessary. 

(1)  Development  of  the  model  itself. 

(1)  Smaller  companies  will  need  simpler  explanation  of  this  system. 

(1 )  Time  to  adequately  review  literature  and  the  National  Research  Council's  Report  Recommendation. 

(1)   Attempting  to  include  PM  1 10  and  cooling  guidelines  in  the  HACCP  program —  they  are  not 
applicable  to  most  products  and  only  confuse  industry  and  regulatory  officers. 

(1 )   Main  meeting  room  space  could  have  been  larger-  more  elbow  space;  overhead  projection  was 
too  low. 

(1 )   Subject  matter  expert  was  not  an  expert  in  the  subject,  but  instead  a  USDA  inspector. 

(1 )  "Nit-picking"  and  obvious  attempt  at  vagueness  exhibited  on  occasion  by  both  USDA  and  industry. 

14.  Please  use  the  rest  of  this  page  to  make  any  additional  comments  that  may  help  us  understand  your 
thoughts  about  the  workshop  and  improve  future  activities. 

(23  comments) 

Workshop-related: 

(5)  Overall  excellent!  The  presentation,  visual  aids  and  USDA's  planning  were  excellent  The  workshop 
was  exceptional  and  productive.  Continue  with  the  workshops;  apply  what  was  learned  here  to 
future  groups.  This  was  a  tremendous  opportunity  to  be  part  of  the  regulatory  process. 

(2)  The  first  7  sections  of  the  program  could  have  been  more  condensed.  Remove  cooling  guidelines, 
PM  110  and  other  non-HACCP  based  guidelines  from  the  HACCP  manual  so  that  they  are  not 
included  in  any  HACCP  plan. 

(2)  Workshop  schedule  should  conclude  in  the  mid-afternoon  to  allow  for  airline  departures.  Schedule 
1-1  /2  hours  for  lunch  each  day  with  this  size  of  a  group.  Without  the  leeway  afforded  by  ending 
sessions  early,  we  would  have  been  too  rushed  since  not  enough  lunch  facilities  were  close-by. 

(1)  Would  like  to  take  a  specific  product  at  the  end  of  the  workshop  and  apply  the  generic  model  to  see 
if  it  works  or  not  or  how  much  modification  it  needs. 

(1)  The  only  issue  was  the  tendency  for  closure  based  on  a  rather  casual  voting  system. 

(1)  Provide  better  definitions  of  the  7  principles  before  asking  the  group  to  define  them  for  the  assigned 
critical  control  points. 

(1)  Steering  committee  selection  should  have  been  based  on  selection  of  persons  most  familiar  with 

HACCP  and  from  different  industry/production  facility  environments  rather  than  voting  for  one  rep 
from  each  group.  The  final  group  selected  in  general  covers  the  above  categories,  although  there  are 
no  representatives  from  small  companies. 

(1 )  Orchestrated  social  functions  might  have  helped  group  dynamics. 
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HACCP-related: 


(1)    I  am  concerned  that  USDA  will  not  be  able  to  demonstrate  quantitative  benefits  of  HACCP  in  pilot 
plants.  They  must  stress  qualitative  benefits  and  consider  a  study  longer  than  6  months. 

(1)    Mention  TQM  and  its  role  in  HACCP.  Teamwork,  training,  education  of  all  employees,  and  operator 
control  are  also  necessary.  A  HACCP  program  without  these  management  tools  stands  naked  and 
is  destined  to  failure. 

(1)   There  is  not  sufficient  incentive  to  be  a  pilot  study  plant. 

Other  ideas: 


(2)   Need  to  have  a  room  large  enough  to  hold  all  attendees  without  obstructions  to  view  {e.g.,  an 
auditorium  setting).  The  group  may  have  been  too  large. 

(1)   A  one-page  summary  outline  of  the  general  approach  used  to  develop  the  generic  HACCP  model 
would  be  of  general  use  to  the  industry. 

(1)   Maintaining  a  semi-official  continuum  of  communication  with /among/between  attendees  after  the 
workshop  probably  has  merit  given  the  long-term  objectives  of  this  important  initiative. 

(1)  USDA  should  consider  the  idea  of  forming  a  veterinary  food  safety  division  of  the  Institute  of  Food 
Technologists  to  professionally  integrate  with  the  technical  community  that  will  ultimately  execute 
the  objectives  of  the  HACCP  initiative. 

(1)   Use  color-coded  name  tags  to  differentiate  among  Participants,  Observers  and  FSIS  staff. 
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SECTION  II—  OBSERVERS 


SUMMARY  OF  OBSERVERS'  RESPONSES  TO 
EVALUATION  QUESTIONNAIRE 


Workshop  for  1 1ACC  I'  Model  1  >cvelopment  for 
Refrigerated  1  nods,  1  ebruary  2b  -  28,  1991 


SUMMARY   OF   OBSERVERS'  RESPONSES 

RESPONDENT  CHARACTERISTICS 

1. 

Which  grouping  below  best  characterizes  your  current  affiliation? 

Meat  &  Poultry  Industry:    8/509*.                         Consultant  to  Meat  &  Poultry  Industry:  1/6% 

Trade  Association:             l/6r/<                          Salad  Industry: 

1/6% 

Educational  Institution:      l/6r,c  Retailer/Wholesaler: 

1/6% 

Government:  3/19c/c 

2. 

If  you  marked  "Meat  &  Poultry  Industry"  in  Question  1,  please  indicate  all  groupings  below  which  best 
characterize  vour  industry  tvpe  and  the  total  production  volume  of  your  establishment(s).  (Low  Volume 
Establishment  =  1  -  59,999  lbs.  produced /Quarter;  Medium  Volume  Establishment  =  60,000-1,000,000  lbs. 
produced /Quarter;  High  Volume  Establishment  =  more  than  1,000,001  lbs.  produced /Quarter] 

Meat  &  Poultry  Industry                Low  Volume  Establishment(s): 

Medium  Volume  Establishment(s): 
High  Volume  Establishment(s): 

n/n<K. 

u/  u  /c 

1/13% 

Poultry  Industry                          Low  Volume  Establishment(s): 

Medium  Volume  Establishment(s): 
High  Volume  Establishment(s): 

ft  ICiGt 
0/0% 

0/0% 
1/13% 

Meat  Industry                             Low  Volume  Establishment(s) 

Medium  Volume  Establishment(s) 
High  Volume  Establishment(s) 

0/0% 
0/0% 
4/50% 

3. 

Days  attended  (mark  all  that  apply):           1  Day:  2/13% 

2  Days:  4/25% 

3  Days: 

10/63% 

4. 

Please  check  one:                                  Observer:  16 

Participant:  0 

HACCP  MODEL  DEVELOPMENT 

Very 

Successful  Successful 

Somewhat  Not 
Successful  Success 

Very 
ful  Unsuccessful 

Don't 
Know 

5. 

Overall,  how  successful  was  the  workshop  in: 

a.  Identifying  Critical  Control  Points  (CCPs)?  2/13%  6/38% 

7/44%  0/0% 

0/0% 

1/6% 

b.  Determining  critical  limits  for  each  CCP?     1/6%  3/19% 

10/63%  1/6% 

0/0% 

1/6% 

c.  Developing  monitoring  plan  activity  for      1/6%  9/56% 
identified  CCPs? 

3/19%  2/13% 

0/0% 

1/6% 

d.  Developing  corrective  action  approaches?   2/13%  8/53% 

4/27%  0/0% 

0/0% 

1/7% 

e.  Developing  record-keeping  systems?         3/20%  5/33% 

5/33%  0/0% 

1/7% 

1/7% 

f.  Developing  verification  procedures?          3/23%  6/46% 

3/23%  0/0% 

0/0% 

1/8% 

g.  Developing  a  generic  HACCP  model?        1/8%  8/62% 

3/23%  0/0% 

0/0% 

1/8% 

NOTE:  Results  are  presented  as  number  of  responses/%  of  responses  for  each  category. 
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DESIGN  AND  OPERATION  Of  WORKSHOP 

Please  rate  the  workshop  in  terms  of: 

a.  Development  of  a  generic  product /process 
description 

b.  Development  of  process  flow  diagram 

c.  Organization  of  the  three  days 

d.  Use  of  small  "break-out"  groups 

e.  Reporting  of  small  "break-out"  group 
activities  to  entire  group 

f.  Opportunity  for  input  from  Observers 

g.  Opportunity  for  input  from  Participants 

h.  Effectiveness  of  facilitators  in  helping 
achieve  workshop  objectives 

i.  Effectiveness  of  subject  matter  experts  in 
helping  achieve  workshop  objectives 

j.    Effectiveness  of  group  leaders  in  helping 
achieve  workshop  objectives 

k.  Effectiveness  of  group  reporters  in  helping 
achieve  workshop  objectives 

1.  Completion  of  workshop  objectives. 


Please  indicate  your  degree  of  agreement 
with  each  of  the  following  statements. 

a.  The  HACCP  model  developed  by  the 
workshop  participants  is  a  viable  frame- 
work for  the  refrigerated  food  industry 
to  implement  on  a  test  basis. 

b.  The  workshop  format  offers  a  sound 
approach  for  producing  a  generic  HACCP 
model  for  other  meat  and  poultry  products. 


to  the  workshop. 

The  workshop  resulted  in  a  useful  gene 
HACCP  model  developed  by  industry 
participants. 


More  than 
Adequate 

Adequate 

Marginally 
Adequate 

Inadequate 

Very 

Inadequate 

Don't 
Know 

0/0  9i 

1  2/R0% 

3/20% 

n/o% 

u/  u  /c 

3/19% 

10/63% 

3/19% 

0/0% 

0/0% 

0/0% 

5/31% 

8/50% 

1/6% 

2/13% 

0/0% 

0/0% 

7/44% 

7/44% 

1/6% 

0/0% 

0/0% 

1/6% 

7/47% 

7/47% 

0/0% 

0/0% 

0/0% 

1/7% 

2/13% 

6/38% 

6/38% 

2/13% 

0/0% 

0/0% 

8/50% 

8/50% 

0/0% 

0/0% 

0/0% 

0/0% 

4/25% 

9/56% 

3/19% 

0/0% 

0/0% 

0/0% 

3/19% 

10/63% 

1/6% 

0/0% 

0/0% 

2/13% 

3/19% 

10/63% 

2/13% 

0/0% 

0/0% 

1/6% 

3/20% 

9/60% 

1/7% 

0/0% 

0/0% 

2/13% 

4/27% 

8/53% 

2/13% 

0/0% 

0/0% 

1/7% 

Strongly 
Agree 

Agree 

Neutral 

Disagree 

Strongly 
Disagree 

Don't 
Know 

1/7% 

7/47% 

4/27% 

0/0% 

0/0% 

3/20% 

2/13% 

6/38% 

6/38% 

0/0% 

0/0% 

2/13%. 

2/13% 

10/63% 

1/6% 

2/13% 

0/0% 

1/6  % 

1/6% 

8/50% 

4/25% 

0/0% 

0/0% 

3/19% 
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WORKSHOP  MATERIALS  (Observers:  Please  skip  Question  8  arid  continue  with  Question  9.) 

Very  Somewhat      Not  Not  At  All 

Useful  Useful         Useful  Useful  Useful 

8.  Overall,  how  useful  were  the  written  materials  for: 

a.  Assessing  finished  product  hazards? 

b.  Assessing  ingredient  hazards? 

c.  Determining  placement  of  CCPs  on  flow 
diagram? 

d.  Determining  critical  limits  for  each  CCP?  OBSERVERS  WERE  INSTRUCTED  TO 

e.  Developing  assigned  CCP  monitoring  plan?  SKIP  THIS  QUESTION. 

f .  Developing  corrective  action  approaches? 

g.  Developing  record-keeping  systems? 

h.  Developing  verification  procedures? 

i.  Developing  a  generic  HACCP  model? 

WORKSHOP  EXPECTATIONS 

9.  Did  the  workshop  accomplish  what  you  expected?  Yes:  12/92%  No:  1/8% 
Please  explain. 

(1 1  comments) 

(4)    Yes.  The  workshop  showed  me  the  step-by-step  process  of  building  a  HACCP  plan.  I  learned 
the  nuts  and  bolts  of  HACCP.  The  basic  approach  was  clear  and  in  place  after  day  one. 

(3)   Both  Yes  and  No.  There  were  too  many  difficulties  involved  in  defining  a  generic  HACCP  model 
on  such  a  broad  category  of  products.  The  generic  model  was  too  generic  and  may  be  of 
questionable  use. 

(2)    Yes.  Developing  the  model  in  a  workshop  was  an  excellent  process,  bringing  industry  and 

government  regulators  together.  Good  exchange —  both  FSIS  and  industry  had  a  chance  to  hear 
where  the  other  stands/what  the  others'  concerns  are. 

(1)   No.  Too  much  time  was  spent  on  explaining  what  USDA  FSIS  has  done  and  too  little  on  specific  goals 
and  objectives:  What  is  expected  of  the  Participants?  Of  the  Observers?  What  is  the  group  trying  to 
create? 

(1)  No.  It  does  not  appear  to  be  feasible  to  develop  a  generic  HACCP  flow  diagram  by  a  committee  of 
60  people. 

10.  Did  the  individual  sessions  accomplish  what  you      Yes  for  Yes  for  No  for 
expected?  Please  explain.                                  All:     6/46%           Some:  7/54%         All:  0/0% 

(4  comments) 

(2)  Yes  for  Some.  Some  sessions  were  not  focussed  and  needed  better  organized  guidelines.  Give  an 
outline  of  steps  to  follow  and  goals  to  be  achieved.  Clearer  directions  or  definitions  of  the  tasks  would 
speed  up  the  process.  Need  better  training  of  facilitators  in  terms  of  group  dynamics/brainstorming. 
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(1 )    Yes.  The  brainstorming  and  the  dialogue  among  industry  Participants  and  FSIS  in  the  break-out 
sessions  were  quite  effective.  Everyone  was  eager  to  participate  and  very  well -prepared. 

(1)  Yes.  The  individual  sessions  were  by  far  the  most  effective  use  of  time.  The  periodic  exposure  to  the 
large  group  evaluation/input  was  definitely  an  asset. 

COMMENTS 

1 1 .  Methods  other  than  a  workshop  format  could  have  been  employed  to  develop  a  generic  HACCP  model. 
Examples  of  options  include  an  FSIS  request  for  (a)  comments  on  an  FSIS-generated  model  published  in 
the  Federal  Register;  (b)  development  of  a  model  by  selected  academicians;  or  (c)  submission  to  FSIS  of 
independently-generated  models.  Please  indicate  your  preference  for  one  of  the  above-listed  options, 
or  describe  any  others  not  mentioned  that  you  would  like  to  be  considered  by  FSIS  in  the  future. 

(12  comments) 

(4)    The  open  forum  of  the  workshop  which  permits  discussion,  challenges  and  development  of 
positions,  concepts  and  principles. 

(4)   Workshop  is  best,  but  have  initial  input  of  a  model  developed  by  a  small  committee  of  FSIS/ 

industry/academia  and  present  it  to  the  entire  workshop.  The  large  workshop  would  then  have  a 
basis  to  work  from.  It  would  be  helpful  if  Participants  understood  the  reason  and  importance  of 
product  descriptions  and  flow  diagram  ahead  of  time.  An  example  would  have  helped —  people 
were  unsure  of  how  to  start. 

(2)  Use  a  combination  of  a,  b  and  c  and  then  submit  the  model  to  workshop  manipulation. 
(1)   Use  c:  submission  to  FSIS  of  independently-generated  models. 

(1)  Use  b:  a  third  party  (academia)  might  reduce  the  bad  feelings  that  FSIS  is  forcing  a  mandatory 
program  on  industry. 

12.  What  was  most  beneficial  to  you  about  the  workshop? 
(13  comments) 

(4)   The  interactions  of  the  knowledgeable  members  of  industry  and  FSIS  as  they  worked  through  building 
the  HACCP  model  were  most  beneficial.  Ideas  were  from  a  good  cross-section  of  the  industry. 

(3)  To  understand  HACCP,  the  complexity  of  HACCP  industry-wide,  and  how  to  create  a  HACCP 
program. 

(2)  The  varied  input  gave  much  more  depth  to  the  development  of  the  model  than  a  one-sided,  single 
company /industry /individual  effort  would  have.  The  in-depth  participation  in  generic  model 
development  served  as  a  better  learning  tool  than  any  course  in  HACCP. 

(1)   The  open  mindedness  of  FSIS. 

(1)   That  it  occured! 

(1)   Being  provided  with  hard  copies  of  each  step  as  it  was  completed  was  extremely  helpful. 

(1)   The  break-out  group  discussion,  even  though  I  was  not  allowed  to  participate.  Opportunity  for 
Observers  to  comment  at  end  of  group  activity  would  be  appropriate,  if  time  permits. 
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13.  What  was  least  beneficial  to  you  about  the  workshop? 
(8  comments) 

(4)  Introductions,  objectives,  workshop  model,  etc.  could  have  been  summarized  in  one  hour.  Get 
this  material  out  ahead  of  time  and  shorten  the  workshop  agenda  or  explanation  thereof. 

(1 )    1  would  have  liked  to  have  seen  a  belter  or  clearer  picture  during  the  first  day's  activities. 

(1 )    Hold  one  or  more  informal  meetings  in  the  evening  even  if  it  is  an  optional  get-together  to 
encourage  interaction  in  a  relaxed  atmosphere. 

(1)   The  room  arrangement — difficult  to  see/hear  participants. 

(1)   Not  being  able  to  participate  actively.  This  was  an  exercise  in  patience  for  me. 

14.  Please  use  the  rest  of  this  page  to  make  any  additional  comments  that  may  help  us  understand  your 
thoughts  about  the  workshop  and  improve  future  activities. 

(29  comments) 

Workshop-related: 

(5)  The  use  of  computers  and  computer-run  overhead  projection  was  key  to  the  success  and  flow  of  the 
workshop.  Electronics  process  was  excellent.  The  audio-visual  portion  of  the  presentation 

was  very  well  done.  Having  an  in-room  photocopy  machine  was  very  useful.  Appreciated 
on-going  updates  of  generic  HACCP  model  by  the  FSIS  staff.  This  was  critical  to  the  success  of 
the  workshop. 

(5)   The  meeting  room  was  inadequate  because  acoustics  and  screens  were  not  adequate  from  the  back  of 
the  room.  It  really  helped  when  we  were  allowed  to  move  up  closer  to  the  front  of  the  room  so  we 
could  see  and  hear  better.  Have  a  speaker  by  the  Observers  and /or  a  microphone  available  for 
Participants  so  their  comments  can  be  heard  by  all.  There  was  too  much  noise  distraction  because  of 
FSIS  people  talking  in  the  back  of  the  room. 

(4)   Observers  should  have  a  complete  notebook/ workbook,  even  if  on  a  daily  loan  basis.  At  least 

provide  Observers  with  the  last  portion  of  the  manual  so  they  could  effectively  follow  the  program 
(i.e.,  the  Principles  and  HACCP  forms). 

(3)   It  would  have  been  easier  to  use  an  existing  model  of  HACCP  from  industry  or  one  developed  by 
National  Marine  Fisheries  Service  as  an  example  on  the  first  day.  Then  it  would  be  clearer  what  CCPs 
are.  Need  to  distinguish  between  a  control  point  and  a  critical  control  point  (CCP).  CCPs  were  not 
adequately  defined.  Also,  some  Participants  couldn't  distinguish  between  quality  assurance  programs 
and  HACCP. 

(1)   It  is  obvious  that  USDA/FSIS  has  done  extensive  preparation  for  HACCP  implementation  to 
assure  industry  and  consumer  activist  groups  that  the  program  is  indeed  a  strengthening  of  FSIS 
regulatory  activities.  Public  relations  preparation  was  well  done. 

(1)   Need  more  training  of  Participants  on  brainstorming  procedures  so  breakout  sessions  would  be  more 
efficient  and  more  ideas  brought  out  and  captured. 

(1)   Suggest  placing  facilitators  at  ends  of  each  table  to  watch  for  raised  hands  during  discussion —  less 
vocal  Participants  were  often  overlooked. 
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(1 )    Need  to  offer  examples  of  monitoring  during  introductory  instructions  to  the  group. 

(1 )    Although  the  overall  size  of  the  group  was  somewhat  unwieldly,  the  breadth  of  information/ 
experience  represented  was  invaluable. 

(1 )    The  smaller  break-out  sessions  take  the  majority  of  the  load  off  the  general  assembly. 

(1 )    Terminology  in  phrases  and  contents  (i.e.,  "process  experts")  should  be  synchronized  or 
coordinated. 

(1)    Well  done. 

HACCP-rclated: 

(1)    Suggest  FSIS  develop  for  the  industry  a  mechanism  to  readily  show  that  a  product  was  produced 
under  an  "approved  HACCP  program."  This  would  be  useful  to  the  food  service  industry  and 
local  government  regulators. 

(1)    Hazard  analysis  in  Principle  1  docs  need  to  be  reviewed  and  changed. 

(1)    CCPs  for  receiving  and  preparation  should  be  optional  for  specific  products. 

(1 )    CCP  for  gas  blend  is  probably  not  necessary. 
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Summary  of  the 
Evaluation  Results  of  the 
Workshop  for  HACCP 
Model  Development 

f 

Fort  Worth,  Texas,  Mai  21  -  23,  1991 


INTRODUCTION 


This  report  presents  the  evaluation  results  from  the  Food  Safety  and  Inspection  Service  (FSIS) 
of  the  United  States  Department  of  Agriculture  (USDA)  workshop  to  develop  a  generic 
model  for  a  hazard  analysis  and  critical  point  (HACCP)  plan  for  cooked  sausage. 
Representatives  from  industry,  trade  associations  and  government  met  in  Fort  Worth, 
Texas,  May  21  -  23, 1991,  for  this  workshop  which  focused  on  cooked  sausage  products. 
The  workshop  was  open  to  the  public  for  observation,  on  a  reserved  space  basis. 

Participants  and  Observers  were  asked  to  complete  a  questionnaire  which  asked  them 
questions  related  to  the  design  and  operation  of  the  workshop,  the  workshop  materials,  the 
process  employed  in  developing  a  generic  HACCP  model,  and  their  views  concerning  the 
workshop  approach  and  any  suggested  improvements.  Of  the  35  Participants  and  25  Observers 
in  attendance,  29  Participants  (83%)  and  18  Observers  (72%)  responded  to  the  questionnaire. 

This  evaluation  activity  is  one  in  a  series  of  on-going  evaluation  tasks  that  are  a  part  of  the 
HACCP  Implementation  Study.  The  results  of  this  evaluation  should  serve  to  help  plan 
subsequent  workshops  and  to  provide  background  information  to  aid  in  future  planning 
decisions  related  to  HACCP. 


REPORT  PLAN 


This  report  is  organized  into  two  sections.  Section  I  presents  the  results  and  summary  of 
Participants'  responses  and  Section  II  presents  those  for  the  Observers.  Results  of  the 
multiple-choice  questions  are  presented  in  a  format  based  on  that  of  the  original  questionnaire. 
The  responses  are  reported  by  number  and  percent  of  responses  for  each  category.  Percentages 
reported  are  based  on  the  number  of  responses  to  each  question  and  do  not  always  total 
100  percent  due  to  arithmetic  rounding. 

Results  of  the  open-ended  questions  are  also  summarized  in  each  Section.  A  content  analysis 
was  performed  for  all  hand-written  comments.  Respondents'  remarks  are  paraphrased 
and  grouped  by  question.  Some  respondents  made  multiple  comments  while  others  made 
none,  thus  the  number  of  comments  per  question  does  not  equal  the  number  of  respondents. 
The  number  of  respondents  who  concurred  with  each  comment  is  listed  parenthetically. 


SECTION  I—  PARTICIPANTS 


SUMMARY  OF  PARTICIPANTS'  RESPONSES 
EVALUATION  QUESTIONNAIRE 


Workshop  fox  HACCP  Model  Development  for 
Cooked  Sausage,  May  21  -  23, 1991 


SUMMARY  OF 

PARTICIPANTS'  RESPONSES 

RESPONDENT  CHARACTERISTICS 

1. 

Which  grouping  below  best  characterizes  your  current  affiliation? 

Meat  &  Poultry  Industry:  25/86% 

Education  Institution: 

1/3% 

Trade  Association:  1/3% 

Research  &  Development  Laboratory:  1/3% 

Casing  Producer  1/3% 

2. 

If  you  marked  "Meat  &  Poultry  Industry"  in  Question  1,  please  indicate  all  groupings  below  which  best 
characterize  your  industry  type  and  the  total  production  volume  of  your  establishments).  [Low  Volume 
Establishment  =  1- 59,999  lbs.  produced/Quarter;  Medium  Volume  Establishment  =  60,000  - 1,000,000  lbs. 
produced /Quarter;  High  Volume-Establishment  =  more  than  1,000,001  lbs.  produced /Quarter] 

Meat  &  Poultry  Industry 

Low  Volume  Establishments): 
Medium  Volume  Establishments): 
High  Volume  Establishments): 

1/4% 
0/0% 
5/20% 

Meat  Industry 

Low  Volume  Establishment 
Medium  Volume  Establishment 
High  Volume  Establishment 

0/0% 

3/12% 

14/56% 

Poultry  Industry 

Low  Volume  Establishments): 
Medium  Volume  Establishments): 
High  Volume  Establishments): 

0/0% 
0/0% 
2/8% 

3. 

Days  attended  (mark  all  that  apply): 

1  Day:  0/0%            2  Days:  0/0% 

3  Days:  29/100% 

4. 

Please  check  one: 

Observer    0            Participant  29 

HACCP  MODEL  DEVELOPMENT 

5. 

Very                        Somewhat  Not          Very  Don't 
Successful  Successful  Successful  Successful  Unsuccessful  Know 

Overall,  how  successful  was  the  workshop  in: 

a.  Identifying  Critical  Control  Points  (CCPs)?  3/10%      10/35%     14/48%  2/7% 

0/0%  0/0% 

b.  Determining  critical  limits  for  each  CCP?    2/7%       12/41%     14/48%  1/3% 

0/0%  0/0% 

c  Developing  monitoring  plan  activity  for     2/7%       18/64%    8/29%  0/0% 
identified  CCPs? 

0/0%  0/0% 

d.  Developing  corrective  action  approaches?  3/10%      21/72%    3/10%  2/7% 

0/0%  0/0% 

e.  Developing  record-keeping  systems? 

2/7%        21/72%     5/17%  1/3% 

0/0%  0/0% 

f .  Developing  verification  procedures? 

2/8%        16/64%     5/20%  2/8% 

0/0%  0/0% 

g.  Developing  a  generic  HACCP  model? 

2/8%        14/54%     8/31%  2/8% 

0/0%  0/0% 

NOTE:  Results  are  presented  as  number  of  responses/%  of  responses  for  each  category. 

-1- 


DESIGN  AND  OPERATION  OF  WORKSHOP 

6. 

Please  rate  the  workshop  in  terms  of: 

More  than 
Adequate 

Adequate 

Marginally 
Adequate 

Inadequate 

V*ry 

Inadequate 

Don't 
Know 

a.  Development  of  a  generic  product  /  process 
description 

2/7% 

23/82% 

2/7% 

1/4% 

0/0% 

0/0% 

b.  Development  of  process  flow  diagram 

X1//Z7* 

i  /iff 

U/Uve 

to  into 
U/U7C 

A /A  fir 

0/0% 

c  Organization  of  the  three  days 

5/17% 

16/55% 

8/28% 

0/0% 

0/0% 

0/0% 

d.  Use  of  small  "break-out"  groups 

10/35% 

15/52% 

4/14% 

0/0% 

0/0% 

0/0% 

e.  Reporting  of  small  "break-out"  group 
activities  to  entire  group 

8/28% 

21/72% 

0/0% 

0/0% 

0/0% 

0/0% 

f .  Opportunity  for  input  from  Observers 

4/15% 

13/48% 

5/19% 

3/11% 

1/4% 

1/4% 

<r    (^OTinrhiTiifv  few  ii>nnt  (tcxtt\  PartiiHTVifiHi 

eL*     VyL/LAJI  lUllliy  lUi  UlUUl                1  Cli  ULiUoilw 

20/69% 

9/31% 

0/0% 

0/0% 

0/0% 

0/0% 

h.  Effectiveness  of  facilitators  in  helping 

skc\\kt*\N*  waiv cVtrtn  nhipfHvpe 

aUUCVC  WUiMlUJp  UURXUVC9 

8/28% 

15/52% 

5/17% 

0/0% 

1/3% 

0/0% 

i.  Effectiveness  of  subject  matter  experts  in 
helping  achieve  workshop  objectives 

3/10% 

13/45% 

9/31% 

3/10% 

1/3% 

0/0% 

j.  Effectiveness  of  group  leaders  in  helping 
achieve  workshop  objectives 

11/38% 

17/59% 

1/3% 

0/0% 

0/0% 

0/0% 

k.  Effectiveness  of  group  reporters  in  helping  9/31% 
achieve  workshop  objectives 

la/OZ7c 

1/370 

1/3  7c 

a /act 

U/U7C 

U/U7C 

1.  Completion  of  workshop  objectives. 

2/7% 

24/86% 

2/7% 

0/0% 

0/0% 

0/0% 

7 
/  • 

Please  indicate  your  degree  of  agreement 
with  each  of  the  following  statements. 

Strongly 
Agree 

Agree 

Neutral 

Disagree 

Strongly 
Disagree 

Don't 
Know 

a.  The  HACCP  model  developed  by  the 
workshop  participants  is  a  viable  frame- 
work for  the  industry  to  implement 
on  a  test  basis. 

1/3% 

19/66% 

5/17% 

4/14% 

0/0% 

0/0% 

b.  The  workshop  format  offers  a  sound 
approach  for  producing  a  generic  HACCP 
model  for  other  meat  and  poultry  products. 

3/10% 

19/66% 

4/14% 

3/10% 

0/0% 

0/0% 

c  The  presence  of  Observers  was  beneficial 
to  the  workshop. 

1/3% 

9/31% 

14/48% 

3/10% 

1/3% 

1/3% 

d.  The  workshop  resulted  in  a  useful  generic 
HACCP  model  developed  by  industry 
participants. 

0/0% 

18/62% 
• 

6/21% 

4/14% 

1/3% 

0/0% 
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WORKSHOP  MATERIALS  (Observers:  Please  skip  Question  8  and  continue  with  Question  9.) 

Very 
Useful 

8.  Overall,  how  useful  were  the  written  materials  for: 

a.  Assessing  finished  product  hazards?  5/17% 

b.  Assessing  ingredient  hazards?  5/17% 

c  Determining  placement  of  CCPs  on  flow  4/14% 
diagram? 

d.  Determining  critical  limits  for  each  CCP?  3/10% 

e.  Developing  assigned  CCP  monitoring  plan?  5/17% 

f .  Developing  corrective  action  approaches?  4/14% 

g.  Developing  record-keeping  systems?  3/10% 

h.  Developing  verification  procedures?  4/14% 

i.  Developing  a  generic  HACCP  model?  4/14% 

WORKSHOP  EXPECTATIONS 

9.  Did  the  workshop  accomplish  what  you  expected? 
Please  explain. 

(23  comments) 

(8)   Yes.  Format  worked  well  to  get  maximum  participation.  It  clarified  several  points  for  me.  I  have  a 
more  in-depth  understanding  of  industry  concerns  and  a  better  understanding  of  exactly  what  FSIS 
expects  from  individual  plants.  Everything  was  covered  in  sufficient  detail  and  written  support 
documents  were  provided  to  facilitate  adaptation  to  our  operations  //back  home."  We  have  our 
generic  model  which  will  not  be  overly  cumbersome  on  even  small  plants.  A  useful  model  was 
developed  albeit  painfully. 

(5)   Yes.  To  a  degree.  A  generic  model  will  come  from  this  workshop  though  I  suspect  it  will  need  to  be 
greatly  refined  when  put  in  place.  More  likely  each  plant  will  need  to  write  their  own  program. 
A  generic  HACCP  program  for  a  specific  product  was  developed.  The  plan  is  very  generic  and  takes 
into  consideration  small  and  large  packers'  needs.  However —  receipt  of  raw  (meat)  materials  must 
be  a  CCP.  I  have  reservations  on  the  applicability  of  the  generic  model,  i.e.,  I  see  the  need  for  control 
points  vs.  critical  control  points  in  the  flow  diagram.  The  model  seems  very  limited  in  scope  and 
might  not  address  all  issues. 

(4)  No.  It  did  not  get  at  the  "gut"  HACCP  issues  (i.e.,  assessing  finished  product  hazards  and  assessing 
ingredient  hazards).  There  was  a  lack  of  depth  in  technical  / analytical  approach  to  hazard  prevention 
and  detection.  The  workshop  moved  too  slowly.  We  needed  less  introduction  on  day  one;  use  3/4  of 
this  day  for  flow  charting,  etc. —  i*.,  cut  to  the  chase.  Spend  more  time  on  flow  chart,  CCPs,  etc. 

(2)   Na  My  expectations  for  this  workshop  were  (1)  to  cooperatively  develop  a  HACCP  model  with 
industry  and  FSIS,  and  (2)  develop  a  generic  model.  Instead,  we  filled  in  the  blanks  of  the  FSIS 
pre-determined  HACCP  model.  This  isn't  necessarily  the  only  valid  HACCP  model  we  could  develop 
for  the  industry.  I  felt  that  FSIS  attempted  to  drive  the  group  to  a  conclusion  FSIS  had  already  reached. 
The  focus  seemed  to  be  on  fitting  CCPs  to  predetermined  areas  of  concern. 


Somewhat 
Useful  Useful 

18/62%  5/17% 
17/59%  6/21% 
16/55%  9/31% 

15/52%  9/31% 

17/59%  6/21% 

16/55%  5/17% 

18/62%  7/24% 

17/59%  7/24% 

17/59%  6/21% 

Yes:  22/76% 


Not  Not  At  All 

Useful  Useful 

0/0%  1/3% 

0/0%  1/3% 

0/0%  0/0% 

2/7%  0/0% 

1/3%  0/0% 

4/14%  0/0% 

1/3%  0/0% 

1/3%  0/0% 

2/7%  0/0% 

No:  7/24% 
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(2)   Ml  The  HACCP  model  is  too  generic  to  satisfy  all  groups.  The  HACCP  document  was  compromised 
to  meet  the  current  practices  of  the  small  firms. 

(2)   No^  We  needed  to  have  prepared  ourselves  better  before  the  meeting.  This  would /could  have  been 
accomplished  with  information  (workshop  book)  sent  to  us  prior  to  the  meeting. 


10.  Did  the  individual  sessions  accomplish  what  you      Yes  for  Yes  for  No  for 
expected?  Please  explain.                                  All:     18/67%          Some:  9/33%         All:  0/0% 

(14  comments) 

(10)  Yes.  The  small  group  was  more  effective  in  that  the  participants  were  on  an  equal  level —  easier  to  be 
heard.  Good  cross-section  of  personalities  and  operations.  The  small  group  concept  is  great  for 
raising  the  participation  level  and  making  rapid  progress.  We  had  good  interaction  and  idea 
exchanges  in  our  small  group.  We  were  able  to  get  input  from  other  individuals  on  their  ideas  of  what 
HACCP  should  accomplish.  These  were  the  best  places  to  get  total  participation  from  all  participants. 
We  stayed  on  track  and  kept  moving.  If  possible  more  should  be  done  in  small  groups.  The  small 
group  work  streamlined  development  procedures  and  could  have  been  used  more  to  develop  other 
parts  of  the  model.  I  would  nave  liked  more  small  group  dynamics  across  all  the  CCPs —  not  just  the 
few  we  were  assigned. 

Yes  for  Some.  Time  spent  on  hazard  analyses  and  assignment  of  CCPs  was  not  efficiently  and 
effectively  used.  Needed  more  discussion  and  assessment  of  actual  risk  based  on  available  scientific 
knowledge  to  develop  CCPs. 

Yes  for  Some.  Some  issues  such  as  metal  detection  and  microtesting  were  surfaced  but  not  put  into  the 
generic  plan  because  of  the  attitude  perceived  in  the  large  group. 

Yes.  Individual  sessions  went  well —  problems  occurred  when  individual  sessions  reported  to  the 
group  as  a  whole. 

COMMENTS 

1 1 .  Methods  other  than  a  workshop  format  could  have  been  employed  to  develop  a  generic  HACCP  model. 
Examples  of  options  include  an  FSIS  request  for  (a)  comments  on  an  FSIS-generated  model  published  in 
the  Federal  Register,  (b)  development  of  a  model  by  selected  academicians;  or  (c)  submission  to  FSIS  of 
independently-generated  models.  Please  indicate  your  preference  for  one  of  the  above-listed  options, 
or  describe  any  others  not  mentioned  that  you  would  like  to  be  considered  by  FSIS  in  the  future. 

(22  comments) 

Concerning  use  of  a  workshop  format: 


(11)  Stay  with  the  workshop  format —  it  was  excellent!  This  is  the  best  method  with  industry  being  able  to 
actively  participate  in  the  development  of  future  FSIS  regulations.  This  is  probably  the  best  method 
for  all  to  air  their  options.  Workshops  attended  by  FSIS  and  interested  industry  people  will  achieve  the 
most  nearly  icleal  balance  of  input  heading  to  development  of  more  equitable  regulations.  Use  of  a 
workshop  combined  with  a  steering  committee  is  a  good  method. 

(1 )  A  review  of  independently-generated  models  would  have  been  useful  as  an  introduction. 

(1)  Use  a  workshop  with  FSIS  and  industry  association  representatives.  Then  submit  that  HACCP  model 
to  the  industry  for  comments.  Bring  FSIS  and  industry  association  representatives  back  together  to 
review  comments  and  finalize  the  HACCP  model. 


Concerning  other  options: 

(4) 

Model  by  industry  is  best  approach;  a  second  choice  would  be  development  of  a  model  by  selected 
academicians  if  industry  can  select  the  academicians.  Development  of  a  model  by  selected 
academicians  would  be  more  non-partial. 

(3) 

Prefer  submission  to  FSIS  of  independently-generated  models. 

(1) 

Definitely  should  not  use  academicians'  suggestions  in  the  development  of  a  HACCP  model. 

(1) 

Prefer  FSIS  to  request  comments  on  an  FSIS-generated  model  published  in  the  Federal  Register. 

12.  What  was  most  beneficial  to  you  about  the  workshop? 

(31  comments) 

(19)  Interaction  with  a  good  cross-section  of  peers  and  FSIS  (both  held  inspectors  and  headquarters 

personnel).  Large  and  small  group  discussions.  Hearing  the  pros  and  cons  on  various  issues.  Gaining 
immediate  feedback  and  learning  concerns  of  other  industry  reps  (both  within  my  industry  and 
within  other  industries).  Meeting  with  large  and  small  packers  and  government  personnel  who  are 
responsible  for  writing  programs.  Insight  into  current  thinking  of  FSIS  regarding  HACCP.  Industry 
participation  was  open  to  all  those  concerned  and  allowed  for  a  wide  variety  of  concerns  to  be  voiced. 
That  industry/FSIS  committment  was  jointly  required.  The  opportunity  to  express  my  views  from 
my  level  (a  low-volume  establishment)  of  the  industry. 

(6) 

The  exposure  to  the  HACCP  theory  and  increased  knowledge  of  HACCP  development  programs.  A 
very  rapid  method  of  getting  "on  line"  with  the  approaches  and  techniques  used  by  major  players  in 
the  industry.  Variation  in  HACCP  concepts  as  perceived  by  industry  personnel.  Talking  with  other 
industry  members  about  their  viewpoints  on  HACCP,  their  companies7  approaches  toward  HACCP 
and  how  they  plan  to  implement  it 

(3) 

Small  group  breakout  sessions/discussions. 

(3) 

Large  and  small  group  discussions. 

(3) 

A  more  thorough  understanding  of  the  HACCP  System  and  how  to  write  a  program  and  implement  it 
at  the  plant  How  chart  and  CCP  identification.  The  workbook  and  the  step-by-step  procedures 
needed  to  formulate  a  HACCP  program. 

13. 

What  was  least  beneficial  to  you  about  the  workshop? 

(22  comments) 

(7) 

Tuesday's  sessions  were  very  general —  the  whole  session  could  have  been  shortened.  The  first 
morning  was  a  waste  of  time.  The  introductions  and  summaries  and  HACCP  overview  were  least 
beneficial.  We  spent  too  much  time  on  HACCP  theory —  this  should  have  been  accomplished  prior 
to  the  workshop. 

(6) 

Large  group  bickering  over  minutia  plus  constant  reminder  of  small  plant  restrictions  and  limitations. 
Allowing  the  large  group  discussion  to  drag.  Too  many  general  comments  that  didn't  help  us 
accomplish  our  objectives.  Too  many  speakers.  Three  days  of  endless  discussion. 
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(2)   The  speakers  spoke  in  a  monotone  voice  and  read  the  materials  from  the  workbook.  They  did  not 
keep  our  interest.  We  don't  need  information  read  to  us. 

(2)   All  the  time  spent  trying  to  clarify  issues  that  could  have  been  quickly  resolved  if  the  subject  matter 
experts  would  have  taken  charge  and  kept  us  on  track.  Subject  matter  experts  basically  did  not 
contribute  in  the  large  group.  Their  perspectives  would  have  been  beneficial. 

(2)   Too  much  time  on  risk  assessment.  Risk  assessment  is  really  not  that  useful  since  we  know  where  the 
risks  are  already  and  will  place  CCPs  where  needed  without  having  to  go  through  risk  assessment. 
Numbering  system  for  hazards  creates  confusion  among  participants. 

(1)   The  use  of  non-meat  company  personnel  as  participants.  They  should  definitely  not  be  on  the  steering 
committee. 

(1 )   Not  having  information,  book  (workshop  book)  beforehand. 
(1)  Nothing.  Every  part  was  extremely  helpful. 

14.  Please  use  the  rest  of  this  page  to  make  any  additional  comments  that  may  help  us  understand  your 
thoughts  about  the  workshop  and  improve  future  activities. 

(26  comments) 

Workshothrelated: 


(4)  I  felt  the  overall  program  started  off  too  slowly.  Start  with  workshop  objectives.  Maybe  the 

introduction  portion  of  program  could  be  shortened  and  an  optional  review  presented  at  the  start  of 
sessions  to  bring  those  participants  who  need  it  up  to  speed  on  HACCP.  Issue  the  workbook  and  other 
information  before  the  meeting.  Two  weeks  would  provide  time  for  participants  to  review  the 
materials  so  they  can  come  to  the  meeting  better  prepared. 

(3)  Discussions  shouldn't  be  allowed  to  get  bogged  down  over  long  periods  of  time  on  tangential 
discussions.  Need  better  leadership  in  closure.  This  workshop  could  be  completed  in  2  days  if  the 
facilitator  would  explain  why  a  certain  CCP  was  included  or  deleted  at  a  previous  workshop.  This 
would  still  allow  the  large  group  to  discuss  the  CCP  and  decide  on  it  but  hopefully,  not  spend  2  hours 
developing  it  and  then  dropping  it  in  the  end. 

(2)  Overall  the  workshop  was  extremely  beneficial  to  all  parties  concerned. 

(2)  I  really  wonder  what  is/has  been  accomplished  by  the  development  of  this  model.  It  is  apparent  that 
FSIS  has  a  clear  idea  of  what  is  expected  in  a  plant  implementation  as  evidenced  by  the  pilot  plant 
program  test  protocol.  During  the  session  it  became  apparent  that  FSIS  has  the  final  product 
developed  thus  our  efforts  were  not  necessary.  Not  so  much  FSIS  model  direction. 

(1)  Be  sensitive  to  busy  periods  in  the  industry  and  have  "workshops"  during  slow  periods.  The  week 
prior  to  "Memorial  Day"  is  one  of  the  busy  periods  for  the  cooked  sausge  industry.  Usually  after 
Christmas  and  prior  to  Easter  are  slow  periods. 

(1 )  Great  graphics  and  material  return]! 

(1)  Simplify  the  worksheet  entitled  "List  Product  Categories  and  Ingredients/'  which  is  too  complex,  to 
address  only  two  groupings  of  ingredients:  "Meat  and  Meat  By-Products"  and  "All  Other."  The 
discussion  to  sort  non-meat  ingredients  into  finer  categories  (binders/ extenders,  spices/flavorings, 
etc.)  is  a  time-waster. 
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(1)    Use  more  breakouts. 


(1)  The  HACCP  document  was  compromised  to  meet  the  current  practices  of  the  small  firms. 
HACCP-related: 


(1)   Need  to  have  better  understanding  of  what  HACCP  is  and  is  not!!!  Confusion  on  process  control 
point,  quality  control  point,  hazard  control  point  and  regulation  control  points.  HACCP  as  presented 
came  off  as  a  TQC  (Total  Quality  Control)  program. 

(1)   As  I  came  into  this  workshop,  I  was  100%  for  HACCP.  As  I  leave,  I'm  not  so  sure  that  this  program 
can  be  developed  generically.  Education  of  inspectors  will  be  required  to  assure  that  knowledgeable 
deviations  from  the  final  plan  can  be  fairly  accomplished. 

(1)   It  is  unfortunate  that  the  development  of  a  comprehensive  HACCP  model  was  impacted,  to  an  extent, 
by  the  fear  that  what  was  developed  would  become  regulatory  requirements  without  a  statistically 
valid,  scientifically-based  approach.  When  the  statement  was  made  that  the  model  would  be  the 
minimum  requirement  that  feeling  was  reinforced. 

Other  ideas: 


(2)   Provide  a  list  of  all  observers,  participants  and  FSIS  personnel  with  titles  and  note  their  functions 
regarding  HACCP.  Include  (telephone  number,  job  title,  location,  company,  etc.) 

(1)  Develop  a  flow  diagram  including  all  present  FSIS  regulations /requirements  included  as  CCPs, 
CPs,  etc.  I  believe  most  CCPs  have  been  defined  by  regulations  but  not  in  a  formal  format  This 
information  would  be  an  asset  to  all  (FSIS  and  industry). 

(1)   Could  have  selected  a  steering  committee  or  group  from  applicants  to  meet  day  prior  to  brainstorm 
and  hammer  out  CCPs  and  present  them  for  comment. 

(1 )  We  needed  more  microphones. 

(1)   Room  great! —  Sound  great!  Excellent  facility  and  hotel. 

(1)   I  now  know  what  it  would  be  like  to  be  a  U.S.  Representative  or  Senator! 

15.  Considering  the  agenda  for  the  first  morning  of  the  workshop,  should  any  of  the  introductory  activities  or 
presentations  be  expanded,  shortened,  or  modified  as  to  content,  to  better  meet  your  needs?  Please  explain. 

(28  comments) 

(19)  Shorten  and  modify  this  section.  Many  of  Tuesday's  session  assumed  no  (little)  knowledge  of  the 
HACCP  program.  Too  much  time  was  spent  on  the  first  1 12  day  on  introductions  and  mechanics. 
This  information  could  be  condensed.  The  first  two  hours  should  be  compressed  to  1 12  hour  to  allow 
more  time  for  more  pertinent  discussions  such  as  discussions  of  flow  charts  and  CCPs.  The  HACCP 
Overview  is  not  needed  unless  it  is  very  much  shortened.  This  is  a  work  session:  we  don't  need  all  the 
extra  information,  etc.  Introductions  of  FSIS  were  too  long  and  not  necessary.  Quickly  introduce 
everyone  then  charge  the  group  with  their  responsibilities  and  explain  what  direction  FSIS's  HACCP 
study  will  take  upon  completion  of  the  workshop. 

(4)  No  need  to  change.  The  pace  was  correct  Need  to  go  ahead  with  this  pace  and  set  the  agenda 
accordingly.  It  was  fine  as  it  was  carried  out 
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(2)   Send  information  packet  containing  the  reading  material  necessary  for  the  1st  morning  to  participants 
and  ask  that  they  become  familiar  with  the  information  before  attending.  Mail  a  current  "briefing7' 
paper  to  the  participants  prior  to  the  workshop  to  bring  them  up-to-date  and  to  free  at  least  1  /  2  of  the 
first  day  for  working  sessions.  Biographies  of  FSIS  personnel  involved  could  be  included. 

(2)   A  review  of  current  HACCP  models  and  a  discussion  of  practical  application  rather  than  theory  may 
have  kept  some  participants  from  confusing  quality  control  with  HACCP  monitoring.  This  review 
could  have  been  limited  to  2  hours.  If  more  information  regarding  the  use  of  the  model  HACCP  plan 
had  been  provided,  the  participants  would  have  been  able  to  give  better  input. 
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SECTION  U—  OBSERVERS 


SUMMARY  OF  OBSERVERS'  RESPONSES 
EVALUATION  QUESTIONNAIRE 


Workshop  for  HACCP  Model  Development  for 
Cooked  Sausage,  May  21  -  23, 1991 


SUMMARY  OF  OBSERVERS'  RESPONSES 

RESPONDENT  CHARACTERISTICS 

1. 

Which  grouping  below  best  characterizes  your  current  affiliation? 

Meat  &  Poultry  Industry:    9/50%                        Consultant  to  Meat  &  Poultry  Industry:  1/6% 

Food  Processor:                2/11%                       Trade  Association: 

1/6% 

Educational  Institution:      2/11%                       Other  Federal  Government: 

1/6% 

Other  Government:  2/11% 

2. 

If  you  marked  "Meat  &  Poultry  Industry"  in  Question  1,  please  indicate  all  groupings  below  which  best 
characterize  your  industry  type  and  the  total  production  volume  of  your  establishment(s).  [Low  Volume 
Establishment  =  1  -  59,999  lbs.  produced /Quarter;  Medium  Volume  Establishment  =  60,000  - 1,000,000  lbs. 
produced /Quarter;  High  Volume  Establishment  =  more  than  1,000,001  lbs.  produced /Quarter] 

Meat  &  Poultry  Industry                Low  Volume  Establishment(s): 

Medium  Volume  Establishment(s): 
High  Volume  Establishment(s): 

0/0% 
0/0% 
5/56% 

Meat  Industry                             Low  Volume  Establishment(s): 

Medium  Volume  Establishment(s): 
High  Volume  Establishment(s): 

0/0% 
0/0% 
4/44% 

Poultry  Industry                          Low  Volume  Establishment(s): 

Medium  Volume  Establishment(s): 
High  Volume  Establishment(s): 

0/0% 
0/0% 
0/0% 

3. 

Days  attended  (mark  all  that  apply):           1  Day:  2/11% 

2  Days:  2/11% 

3  Days: 

14/78% 

4. 

Please  check  one:                                  Observer:  18 

Participant:  0 

HACCP  MODEL  DEVELOPMENT 

5. 

Very 

Successful  Successful 

Overall,  how  successful  was  the  workshop  in: 

Somewhat  Not  Very 
Successful  Successful  Unsuccessful 

Don't 
Know 

a.  Identifying  Critical  Control  Points  (CCPs)?  0/0%  13/72% 

2/11%  2/11% 

1/6% 

0/0% 

b.  Detenriining  critical  limits  for  each  CCP?     1/6%  7/39% 

6/33%  3/17% 

1/6% 

0/0% 

c.  Developing  monitoring  plan  activity  for      0/0%  11/61% 
identified  CCPs? 

4/22%  2/11% 

1/6% 

0/0% 

d.  Developing  corrective  action  approaches?   1/6%  7/39% 

7/39%  2/11% 

1/6% 

0/0% 

e.  Developing  record-keeping  systems?         2/11%  9/50% 

5/28%  1/6% 

1/6% 

0/0% 

f.  Developing  verification  procedures?          1/6%  8/47% 

5/29%  2/12% 

1/6% 

0/0% 

g.  Developing  a  generic  HACCP  model?        2/11%  9/50% 

5/28%  1/6% 

1/6% 

0/0% 

NOTE:  Results  are  presented  as  number  ofresponsesl%  of  responses  for  each  category. 
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DESIGN  AND  OPERATION  OF  WORKSHOP 

6. 

Please  rate  the  workshop  in  terms  of: 

More  than 
Adequate 

Adequate 

Marginally 
Adequate 

Inadequate 

Very 

Inadequate 

Don't 
Know 

a. 

Development  of  a  generic  product /process 
description 

4/22% 

9/50% 

4/22% 

1/6% 

0/0% 

0/0% 

b. 

Development  of  process  flow  diagram 

2/11% 

12/67% 

3/17% 

0/0% 

0/0% 

1/6% 

c. 

Organization  of  the  three  days 

5/29% 

7/41% 

5/29% 

0/0% 

0/0% 

0/0% 

d. 

Use  of  small  "break-out"  groups 

6/33% 

8/44% 

3/17% 

0/0% 

0/0% 

1/6% 

e. 

Reporting  of  small  "break-out"  group 
activities  to  entire  group 

3/17% 

11/61% 

3/17% 

1/6% 

0/0% 

0/0% 

f. 

Opportunity  for  input  from  Observers 

0/0% 

7/44% 

5/31% 

2/13% 

2/13% 

0/0% 

g- 

Opportunity  for  input  from  Participants 

7/39% 

10/56% 

0/0% 

1/6% 

0/0% 

0/0% 

h. 

Effectiveness  of  facilitators  in  helping 
achieve  workshop  objectives 

2/11% 

12/67% 

3/17% 

0/0% 

1/6% 

0/0% 

i. 

Effectiveness  of  subject  matter  experts  in 
helping  achieve  workshop  objectives 

0/0% 

6/33% 

6/33% 

1/6% 

1/6% 

4/22% 

j- 

Effectiveness  of  group  leaders  in  helping 
achieve  workshop  objectives 

1/6% 

13/72% 

2/11% 

0/0% 

0/0% 

2/11% 

k. 

Effectiveness  of  group  reporters  in  helping 
achieve  workshop  objectives 

2/11% 

a-J  X  X  ifJ 

14/78% 

1/6% 

0/0% 

0/0% 

1/6% 

1. 

Completion  of  workshop  objectives. 

2/11% 

11/61% 

2/11% 

1/6% 

1/6% 

1/6% 

7 

Please  indicate  your  degree  of  agreement 
with  each  of  the  following  statements. 

Strongly 
Agree 

Agree 

Neutral 

Disagree 

Strongly 
Disagree 

Don't 
Know 

a. 

The  HACCP  model  developed  by  the 
workshop  participants  is  a  viable  frame- 
work for  the  industry  to  implement 
on  a  test  basis. 

2/12% 

10/59% 

1/6% 

2/12% 

2/12% 

0/0% 

b. 

The  workshop  format  offers  a  sound 
approach  for  producing  a  generic  HACCP 
model  for  other  meat  and  poultry  products. 

4/22% 

8/44% 

3/17% 

2/11% 

1/6% 

0/0% 

c. 

The  presence  of  Observers  was  beneficial 
to  the  workshop. 

5/28% 

5/28% 

4/22% 

1/6% 

2/11% 

1/6% 

d. 

The  workshop  resulted  in  a  useful  generic 
HACCP  model  developed  by  industry 
participants. 

3/17% 

10/56% 

1/6% 

2/11% 

2/11% 

0/0% 
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WORKSHOP  MATERIALS  (Observers:  Please  skip  Question  8  and  continue  with  Question  9.) 

Very  Somewhat     Not  Not  At  All 

Useful  Useful        Useful  Useful  Useful 

8.  Overall,  how  useful  were  the  written  materials  for: 

a.  Assessing  finished  product  hazards? 

b.  Assessing  ingredient  hazards? 

c.  Determining  placement  of  CCPs  on  flow 
diagram? 

d.  Determining  critical  limits  for  each  CCP?  OBSERVERS  WERE  INSTRUCTED  TO 

e.  Developing  assigned  CCP  monitoring  plan?  SKIP  THIS  QUESTION. 

f.  Developing  corrective  action  approaches? 

g.  Developing  record-keeping  systems? 

h.  Developing  verification  procedures? 

i.  Developing  a  generic  HACCP  model? 

WORKSHOP  EXPECTATIONS 

9.  Did  the  workshop  accomplish  what  you  expected?  Yes:  13/72%  No:  5/28% 
Please  explain. 

(15  comments) 

(5)    Yes.  A  generic  HACCP  model  was  developed  which  should  assist  in  process  standardization  for  this 
industry.  This  generic  model  will  serve  as  the  foundation  on  which  to  build  plant-specific  plans. 
FSIS  representatives  worked  very  closely  with  members  of  the  industry  to  develop  a  generic  HACCP 
plan  everyone  could  agree  upon  and  live  with. 

(2)   Yes.  This  format  should  be  used  in  the  future  for  major  initiatives  and  / or  changes  in  the  inspection 
system.  The  workshop  format  facilitates  communication  and  cooperation  between  regulatory  agencies 
and  industry.  Industry  can't  say  it  doesn't  have  a  voice  or  input  in  new  initiatives.  HACCP  principles 
and  the  process  for  model  construction  were  covered  very  well.  Besides  the  hazard  evaluation 
which  needs  to  be  re-thought  in  terms  of  chemical  and  physical  hazards,  the  exercise  provided  in- 
dustry and  the  regulatory  agency  a  common  approach  to  the  implementation  of  HACCP  on  cooked 
sausage. 

(2)    Yes.  As  an  observer,  it  gave  me  a  better  undertanding  of  HACCP  and  how  it  will  work.  It  broadened 
the  scope  of  alternatives  each  processor  can  apply  to  their  particular  products. 

(2)    Yes.  But  qualified —  in  many  of  the  discussions  and  decisions,  there  was  a  complete  absence  of  any 
science  being  used.  I  do  not  understand  what  is  "science-based"  about  this  program  (the  generic 
model). 

(1)   Na  I  do  not  believe  the  developed  model  will  stand  up  in  real  plant  operations,  however,  the  workshop 
was  an  excellent  training  effort.  Most  of  the  people  in  attendance  possessed  a  high  degree  of  technical 
knowledge  but  this  level  of  knowledge  is  not  representative  of  cooked  sausage  manufacturers. 
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(1 )    No.  The  CCPs  chosen  do  not  all  reflect  food  safety.  Need  to  separate  food  safety  and  quality.  The 

industry  participants  do  not  all  have  expertise  or  understand  the  HACCP  program  enough  to  come  up 
with  an  effective  program. 

(1)    No.  Prior  to  this  workshop  I  did  not  realize  the  complete  format  was  pre-determined  by  FSIS.  Industry 
only  filled  in  the  blanks  in  the  forms. 

(1)    Na  It  only  pertained  to  a  very  limited  product  line.  I  would  have  rather  seen  a  more  general  plan  for 
all  cooked  sausages. 


10.   Did  the  individual  sessions  accomplish  what  you      Yes  for  Yes  for  No  for 

expected?  Please  explain.  All:     6/38%  Some:  9/57%         All:  1/6% 

(10  comments) 

(5)    Yes.  The  small  group  activity  was  more  productive  (time)  than  the  large  group.  Participation  of  group 
members  ensured  all  types  of  operations  were  represented  in  developing  HACCP  guidelines  and 
requirements.  Each  session  was  a  good  mental  exercise.  Individual  participants  worked  well 
together;  all  participated.  Facilitators  kept  the  group  going  in  the  right  direction  at  a  good  pace.  A 
HACCP  model  was  established  through  the  input  of  these  individual  sessions. 

(2)  Yes  for  Some.  In  many  of  the  discussions  and  decisions,  there  was  a  complete  absence  of  any  science 
being  used.  Hazard  assessment  has  to  be  refined  in  order  to  be  very  useful  but  even  this  session  has 
been  interesting  in  terms  of  the  interaction  among  the  participants. 

(2)    Yes  for  Some.  Under  the  "definitions"  of  the  workshop,  as  stated  on  Tuesday,  some  sessions 

approximated  the  desired  or  anticipated  results.  Identifying  CCPs  and  assigning  categories  of  CCPs 
went  slowly  at  times.  The  sessions  were  generally  good,  however,  Wednesday  afternoon  proceeded 
very  slowly.  I  had  expected  the  general  sessions  to  brainstorm  ideas  for  the  small  group  sessions 
to  discuss  what  would  work  and  what  should  be  eliminated. 

(1)    Yes  for  Some.  Observers  should  have  a  few  minutes  to  comment  after  each  major  agenda  item. 


COMMENTS 

11.  Methods  other  than  a  workshop  format  could  have  been  employed  to  develop  a  generic  HACCP  model. 
Examples  of  options  include  an  FSIS  request  for  (a)  comments  on  an  FSIS-generated  model  published  in 
the  Federal  Register;  (b)  development  of  a  model  by  selected  academicians;  or  (c)  submission  to  FSIS  of 
independently-generated  models.  Please  indicate  your  preference  for  one  of  the  above-listed  options, 
or  describe  any  others  not  mentioned  that  you  would  like  to  be  considered  by  FSIS  in  the  future. 

(17  comments) 

Concerning  use  of  a  workshop  format: 

(6)   This  workshop  was  very  successful.  The  training /educational  approach  underlying  this  workshop 
should  be  thoroughly  considered  for  future  efforts.  To  incorporate  a  major  change  in  industry 
regulations,  workshops  are  the  only  method.  All  other  methods  do  not  allow  industry  to  agree  on 
formats.  Submission  of  independently-generated  models  may  work  but  doesn't  allow  industry  the 
opportunity  to  discuss  and  arrive  at  a  majority  decision.  This  workshop  format  is  excellent  to  rapidly 
come  up  with  a  model  that  would  be  accepted  by  at  least  the  major  stake  holders  in  the  industry.  The 
open  workshop  format  is  not  only  the  most  efficient  for  all  involved  but  also  promotes  positive  feel- 
ings between  the  industry  and  government. 
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(1 )    More  small  group  activity  prior  to  large  group  discussion. 

(1)  Drop  the  current  risk  assessment  since  it  plays  no  role  without  considering  what  you  can  do  to  control 
the  process. 

Concerning  other  options: 

(3)    Other  options  include:  (1)  submitting  the  generic  HACCP  model  to  a  much  smaller  group  of  industry 
experts  (processors  would  still  need  to  put  together  their  own  programs);  (2)  running  a  pilot  in  one 
of  a  few  plants  and  circulating  the  proposed  developed  model  to  the  rest  of  industry  for  comments; 
and  (3)  using  academicians  or  independently-generated  models  (option  b  or  c),  adding  input  from 
HACCP  experts,  and  finalizing  the  process  with  a  Delphi  technique  to  formulate  a  HACCP  model. 

(2)  C  (independently-generated  models  submitted  to  FSIS)  would  be  my  preference  on  the  above-listed 
options. 

(2)    Of  the  three  choices,  I  would  reluctantly  choose  b  and  pray  that  a  well-balanced  group  of  academicians 
would  approach  the  job  in  an  unbiased  manner.  I  like  a  mix  of  academicians  and  industry  personnel 
to  develop  future  models. 

(1)   I  would  prefer  an  FSIS-generated  model  be  published  in  the  Federal  Register  for  comments. 

(1)   The  Federal  Register  is  a  slow  method  and  not  everyone  has  access  to  it. 


12.  What  was  most  beneficial  to  you  about  the  workshop? 
(19  comments) 

(8)    Interaction  with  FSIS.  Gaining  an  understanding  of  the  "process"  FSIS  is  using  in  developing  HACCP. 
Ability  to  visit  in  a  more  social  setting  with  FSIS  members.  Exchange  of  ideas  and  information  with 
industry  and  FSIS.  As  an  observer  I  gained  the  most  from  hearing  the  different  methods  of  control 
that  different  plants  implement.  Opportunity  for  the  industry  to  give  input.  The  industry  concerns 
and  the  way  these  were  addressed  by  the  facilitator  and /or  FSIS  representatives. 

(4)   The  development  of  procedures  to  fill  out  the  form  "Industry  Workshop  HACCP  Model."  Procedures 
followed  to  develop  a  HACCP  plan,  rather  than  the  finished  product  (the  actual  plan).  The 
development  of  the  flow  chart  and  selecting  CCPs. 

(3)   Discussion  groups  were  very  good.  Glad  to  see  discussion  by  most  of  the  participants. 

(3)   The  HACCP  model.  It  will  bring  about  some  uniformity  in  the  way  industry  will  understand  HACCP 
principles  and  will  serve  as  a  basis  for  consistent  implementation  at  the  plant  level. 

(1 )   I  liked  getting  all  the  handouts. 
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13.    What  was  least  beneficial  to  you  about  the  workshop? 
(13  comments) 

(4)    The  first  morning.  The  laborious  introductions  were  not  needed.  The  introductory  session  was  a  bit 
too  long  and  repetitive  in  some  aspects.  This  session  could  be  polished  to  save  time  for  better  use. 
Introduction  concerning  previous  workshops  and  additional  information  not  related  to  this  workshop. 

(3)    Listening  to  participants  argue  excessively  over  insignificant  issues.  This  should  have  been  halted 
before  wasting  valuable  time.  Perhaps  FSIS  needs  to  step  in  when  participants  have  hashed  an  issue 
out  long  enough  and  the  discussion  is  going  nowhere. 

(2)    Preconceived  HACCP  outline  by  FSIS  personnel,  particularly  with  raw  materials  at  receiving.  This 
is  a  make-work,  "done  deal"  for  industry. 

(2)    Industry  tendency  to  keep  model  very  generic  so  that  they  will  not  be  "nailed  down"  in  future 

programs.  The  model  could  be  more  specific  and  therefore  would  be  more  useful.  Determination  of 
CCPs  was  reduced  below  a  level  where  I  feel  comfortable  that  food  safety  would  be  guaranteed. 
Industry  looked  at  CCPs  from  an  economic  viewpoint  instead  of  a  consumer-safety  standpoint. 

(1)   I  would  suggest  the  HACCP  model  has  been  overly  considerate  of  the  capabilities  of  the  smaller 
processors  thus  limiting  the  effectiveness  of  the  program.  Micro  and  pathogenic  monitoring  is 
needed.  The  model  needs  a  lot  of  work  to  make  it  effective  to  assure  safe  product.  Small  processors 
possibly  should  not  be  in  business  if  they  cannot  afford  verification  steps  to  produce  safe  product. 

(1)   The  procedure  to  develop  Hazards  A  -  F  Categories  I-VI  is  a  waste  of  time  and  confusing,  however, 
these  same  worthwhile  principles  are  described  clearly  in  the  FSIS  report  HACCP  Principles  for  Food 
Production. 


14.  Please  use  the  rest  of  this  page  to  make  any  additional  comments  that  may  help  us  understand  your 
thoughts  about  the  workshop  and  improve  future  activities. 

(22  comments) 

Workshop-related: 


(7)   The  workshop  was  prepared  and  managed  in  a  very  professional  way.  Time  management  was  a  key 
element  considering  the  amount  of  work  the  participants  had  to  complete.  The  group  dynamic  was 
excellent.  Participation  was  well  balanced  among  industry  participants.  The  workshop  was  very 
positive.  Thank  you  so  much  for  allowing  us  the  chance  to  observe  and  learn  from  these  workshops . 
The  information  and  experience  we  gathered  is  invaluable.  Generally  speaking,  these  sessions  are 
profitable  due  to  the  exchange  of  information,  data  and  ideas.  It  was  very  well-planned!  I  applaud 
the  hotel  selection! 

(4)   In  the  future  communicate  sufficiently  to  make  sure  industry  sends  the  individuals  with  sufficient 
expertise  to  make  solid  contributions.  Participants  often  could  not  distinguish  between  a  control  point 
(product  quality/market  driven  characteristic)  and  a  critical  control  point  (product  safety).  Who 
selected  the  participants?  This  model  may  be  classified  as  "modified  PBIS"  but  it  surely  isn't  HACCP 
by  any  stretch  of  the  imagination.  Apparently,  there  is  a  misunderstanding,  or  lack  of  knowledge, 
as  to  the  difference  between  a  control  point  and  a  critical  control  point.  The  participants  identified 
control  points  mostly  but  are  out  of  touch  with  CCPs. 
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The  makeup  of  the  group  docs  not  represent  the  industry  so  you  may  have  been  preaching  to  the  choir. 
I  recommend  that  you  refrain  from  developing  regulations  based  on  this  workshop.  Need  to  en- 
courage "more  representation  of  smaller  plants.  They  sure  need  to  be  at  these  workshops."  More  sites 
for  these  workshops  might  facilitate  more  plants  being  represented. 

I  do  not  understand  the  use  of  "consensus"  when  one  side  won  an  election  over  the  other  side  with  a 
vote  of  18  to  16.  A  definition  for  the  term  "consensus"  should  be  agreed  upon  before  any  voting 
takes  place. 

The  workshop  provided  a  great  mental  exercise.  Although  most  of  the  participants  were  already  very 
knowledgeable,  they  probably  did  get  a  good  understanding  of  how  different  people  view  process 
control  and  therefore,  how  difficult  it  is  for  FSIS  to  standardize  inspection  procedures. 

I  was  very  disappointed  that  the  workshop  was  structured  to  prohibit  any  participation  from  observers. 
There  was  expertise  in  the  observers'  group  that  could  have  added  input  and  clarified  some  of  the 
discussion. 

Steering  committee  membership  should  have  been  based  on  expertise  instead  of  availability.  Some  of 
the  committee  members  did  not  demonstrate  their  expertise  sufficiently. 

Although  the  hazard  analysis  risk  assessment  was  not  integrated  into  the  model,  it  was  beneficial  in 
bringing  out  important  points. 

The  regional  coordinators  were  most  helpful.  They  gave  a  hands-on  approach  to  HACCP  in  response 
to  my  many  questions! 


HACCP-related: 


(1)   From  a  public  relations  standpoint  we  should  change  the  name  and  eliminate  the  word  "hazards." 

(1)   Explain  FSIS's  views  on  how  the  small  plants  will  implement  a  HACCP  program.  They  see  it  as  being 
too  expensive.  They  have  no  QC  program  or  lab  for  testing.  All  plants  are  different.  How  do  you 
implement  a  uniform  HACCP  program  to  both  large  and  small  plants?  Not  everyone  is  on  the  same 
level  playing  field. 


15.   Considering  the  agenda  for  the  first  morning  of  the  workshop,  should  any  of  the  introductory  activities  or 
presentations  be  expanded,  shortened,  or  modified  as  to  content,  to  better  meet  your  needs?  Please  explain. 

(18  comments) 

(11)  Most  of  the  first  morning's  agenda  could  be  shortened/ streamlined  without  losing  any  value  or 

content.  The  laborious  introductions  were  not  needed.  This  time  could  be  used  for  better  explaining 
the  workshop  agenda.  The  program  was  too  repetitious  and  moved  too  slowly.  FSIS  leaders  should 
have  kept  the  discussions  moving  faster.  Omit  the  history  concerning  previous  workshops.  Much  of 
that  information  was  in  the  workshop  packet.  Maybe  there  was  too  much  emphasis  in  explaining  the 
mechanics  of  the  workshop.  The  HACCP  overview  was  a  little  bit  "rushed."  More  time  could  be  used 
to  explain  the  HACCP  model,  CPs,  CCPs,  monitoring,  etc. 
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(4)    The  length  of  the  presentation  is  just  right.  The  current  agenda  is  acceptable.  New  participants  and 
observers  need  this  introduction  although  it's  a  little  tedious  for  those  who  have  attended  prior 
meetings.  The  introductory  activities  arc  important  in  the  explanation  of  HACCP  and  its  importance 
in  an  inspection  method.  The  only  thing  I  would  suggest  is  perhaps  fitting  in  a  brief  summary  of  the 
2-ycar  study  FSIS  has  undertaken —  how  they  started,  what  they've  done,  where  they're  going,  etc. 
I  don't  think  everyone  appreciates  just  how  much  time,  work,  and  effort  has  gone  into  this.  Not  only 
would  this  allow  the  FSIS  folks  to  "brag"  a  little  (as  well  they  should!)  but  by  increasing  industry's 
awareness  of  all  that  has  gone  into  the  study,  perhaps  the  participants  would  be  a  little  more 
appreciative  and  take  the  workshops  more  seriously. 

(2)    It  was  obvious  that  some  participants  had  not  read  the  background  document  provided  to  them  prior 
to  the  meeting.  For  example,  the  group  should  have  had  more  knowledge  of  HACCP  before  the 
workshop  since  much  of  the  discussion  was  related  to  misunderstandings  about  CCPs  &  CPs.  For 
future  workshops,  individuals  not  on  the  expected  wavelength  should  be  encouraged  (politely)  to 
read  the  workshop  materials  the  first  night  of  the  session  in  order  to  fully  benefit  from  the  exercise. 
The  time  saved  could  be  allocated  to  the  model  development. 

(1)    I  liked  the  self-introductions  by  Participants  and  Observers. 
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Summary  of  the 
Evaluation  Results  of  the 
Workshop  for  HACCP 
Model  Development 

Atlanta,  Georgia,  August  27  -  29,  1991  Jj 
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INTRODUCTION 


This  report  presents  the  evaluation  results  from  the  Food  Safety  and  Inspection  Service  (FSIS) 
of  the  United  States  Department  of  Agriculture  (USD A)  workshop  to  develop  a  generic 
model  for  a  hazard  analysis  and  critical  point  (HACCP)  plan  for  poultry  slaughter. 
Representatives  from  industry/  trade  associations  and  government  met  in  Atlanta,  Georgia, 
August  27  -  29, 1991,  for  this  workshop  which  focused  on  young  chicken  slaughter. 
The  workshop  was  open  to  the  public  for  observation,  on  a  reserved  space  basis. 

Participants  and  Observers  were  asked  to  complete  a  questionnaire  which  asked  them 
questions  related  to  the  design  and  operation  of  the  workshop,  the  workshop  materials,  the 
process  employed  in  developing  a  generic  HACCP  model,  and  their  views  concerning  the 
workshop  approach  and  any  suggested  improvements.  Of  the  43  Participants  and  54  Observers 
in  attendance,  41  Participants  (95%)  and  29  Observers  (54%)  responded  to  the  questionnaire. 

This  evaluation  activity  is  one  in  a  series  of  on-going  evaluation  tasks  that  are  a  part  of  the 
HACCP  Implementation  Study.  The  results  of  this  evaluation  should  serve  to  help  plan 
subsequent  workshops  and  to  provide  background  information  to  aid  in  future  planning 
decisions  related  to  HACCP. 


REPORT  PLAN 


This  report  is  organized  into  two  sections.  Section  I  presents  the  results  and  summary  of 
Participants'  responses  and  Section  II  presents  those  for  the  Observers.  Results  of  the 
multiple-choice  questions  are  presented  in  a  format  based  on  that  of  the  original  questionnaire. 
The  responses  are  reported  by  number  and  percent  of  responses  for  each  category.  Percentages 
reported  are  based  on  the  number  of  responses  to  each  question  and  do  not  always  total 
100  percent  due  to  arithmetic  rounding. 

Results  of  the  open-ended  questions  are  also  summarized  in  each  Section.  A  content  analysis 
was  performed  for  all  hand-written  comments.  Respondents' remarks  are  paraphrased 
and  grouped  by  question.  Some  respondents  made  multiple  comments  while  others  made 
none,  thus  the  number  of  comments  per  question  does  not  equal  the  number  of  respondents. 
The  number  of  respondents  who  concurred  with  each  comment  is  listed  parenthetically. 


SECTION  I—  PARTICIPANTS 


SUMMARY  OF  PARTICIPANTS'  RESPONSES  TO 
EVALUATION  QUESTIONNAIRE 


Workshop  for  HACCP  Model  Development  for 
Poultry  Slaughter,  August  27  -  29, 1991 


SUMMARY  OF  PARTICIPANTS'  RESPONSES 


RESPONDENT  CHARACTERISTICS 


1 .    Which  grouping  below  best  characterizes  your  current  affiliation? 

Meat  &  Poultry  Industry:  38/93% 
Education  Institution:  2/5% 
Trade  Association:  1/2% 


2.    If  you  marked  "Meat  &  Poultry  Industry"  in  Question  1,  please  indicate  all  groupings  below  which  best 
characterize  your  industry  type  and  the  total  production  volume  of  your  establishments).  [Low  Volume 
Establishment » 1  -  59,999  lbs.  produced/Quarter;  Medium  Volume  Establishment  =  60,000  - 1,000,000  lbs. 
produced /Quarter;  High  Volume  Establishment  =  more  than  1,000,001  lbs.  produced /Quarter] 


Meat  &  Poultry  Industry 


Poultry  Industry 


Low  Volume  Establishments):  0/0% 

Medium  Volume  Establishments):  7/18% 

High  Volume  Establishments);  0/0% 

Unspecified  Volume:  6/16% 

Low  Volume  Establishments):  0/0% 

Medium  Volume  Establishments):  3/8% 

High  Volume  Establishments):  20/53% 

Unspecified  Volume:  2/5% 


/ 


3.  Days  attended  (mark  all  that  apply): 

4.  Please  check  one: 

HACCP  MODEL  DEVELOPMENT 


IDay:  3/7% 
Observer  0 


2  Days:  1/3% 
Participant  41 


3  Days:  37/90% 


Overall,  how  successful  was  the  workshop  in: 

Very 

Successful  Successful 

Somewhat  Not 
Successful  Successful 

Very  Don't 
Unsuccessful  Know 

a.  Identifying  Critical  Control  Points  (CCPs)?  1/2% 

16/39% 

20/49% 

1/3% 

3/7% 

0/0% 

b.  Determining  critical  limits  for  each  CCP? 

1/2% 

22/54% 

11/27% 

4/10% 

3/7% 

0/0% 

c  Developing  monitoring  plan  activity  for 
identified  CCPs? 

2/5% 

20/50% 

15/38% 

1/2% 

2/5% 

0/0% 

d.  Developing  corrective  action  approaches? 

2/5% 

19/48% 

14/35% 

3/8% 

2/5% 

0/0% 

e.  Developing  record-keeping  systems? 

5/13% 

20/50% 

11/28% 

3/8% 

1/3% 

0/0% 

f.  Developing  verification  procedures? 

3/9% 

18/55% 

8/24% 

3/9% 

1/3% 

0/0% 

g.  Developing  a  generic  HACCP  model?. 

3/8% 

15/40% 

15/40% 

3/8% 

2/5% 

0/0% 

NOTE:  Results  are  presented  as  number  of  responses/%  of  responses  for  each  category. 
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DESIGN  AND  OPERATION  OF  WORKSHOP 

6. 

Please  rate  the  workshop  in  terms  of: 

More  than 
Adequate 

Adequate 

Marginally 

Adequate  Inadequatt 

Very 
(  Inadequate 

Don't 
Kjura? 

a.  Development  of  a  generic  product/process  2/5% 
description 

24/59% 

10/24% 

2/5% 

3/7% 

0/0% 

b.  Development  of  process  flow  diagram 

9/22% 

27/66% 

1/2% 

0/0% 

c  Organization  of  the  three  davs 

13/32% 

20/49% 

7/17% 

0/0% 

1/2% 

0/0% 

d.  Use  of  small  "break-out"  groups 

13/32% 

25/61% 

1/2% 

1/2% 

1/2% 

0/0% 

e.  Reporting  of  small  "break-out"  group 
activities  to  entire  group 

11/27% 

27/66% 

2/5% 

0/0% 

1/2% 

0/0% 

f.  Opportunity  for  input  from  Observers 

4/10% 

20/49% 

4/10% 

8/20% 

3/7% 

2/5% 

g.  Opportunity  for  input  from  Participants 

20/49% 

17/42% 

3/7% 

1/2% 

0/0% 

0/0% 

h.  Effectiveness  of  facilitators  in  helping 
achieve  workshop  objectives 

9/23% 

21/53% 

5/13% 

4/10% 

1/3% 

0/0% 

L  Effectiveness  of  subject  matter  experts  in 
helping  achieve  workshop  objectives 

5/12% 

20/49% 

9/22% 

4/10% 

3/7% 

0/0% 

j.  Effectiveness  of  group  leaders  in  helping 
achieve  workshop  objectives 

8/20% 

30/73% 

1/2% 

1/2% 

1/2% 

0/0% 

k.  Effectiveness  of  group  reporters  in  helping  12/29% 
achieve  workshop  objectives 

25/61% 

4/10% 

0/0% 

0/0% 

0/0% 

1.  Completion  of  workshop  objectives. 

3/8% 

24/62% 

8/21% 

2/5% 

2/5% 

0/0% 

7. 

Please  indicate  your  degree  of  agreement 
with  each  of  the  following  statements. 

Strongly 
Agra 

Agra 

Neutml 

Disagree 

Disagree 

Do*'* 
Xwtm? 

a.  The  HACCP  model  developed  by  the 
workshop  participants  is  a  viable  frame- 
work for  the  industry  to  implement 
on  a  test  basis. 

3/8% 

22/55% 

10/25% 

2/5% 

3/8% 

0/0% 

b.  The  workshop  format  offers  a  sound 
approach  for  producing  a  generic  HACCP 
model  for  other  meat  and  poultry  products. 

3/8% 

24/60% 

7/18% 

4/10% 

2/5% 

0/0% 

c  The  presence  of  Observers  was  beneficial 
to  the  workshop. 

2/5% 

13/32% 

15/37% 

7/17% 

1/2% 

3/7% 

&  The  workshop  resulted  in  a  useful  generic 
HACCP  model  developed  by  industry 
participants. 

3/8% 

19/48% 

11/28% 

4/10% 

3/8% 

0/0% 
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WORKSHOP  MATERIALS  (Observers:  Please  skip  Question  8  and  continue  with  Question  9.) 

Useful 


Very 
Useful 


Somewhst 
Useful 


8.    Overall,  how  useful  were  the  written  materials  for 

a.  Assessing  finished  product  hazards? 

b.  Assessing  ingredient  hazards? 

c  Determining  placement  of  CCPs  on  flow 
diagram? 

d  Determining  critical  limits  for  each  CCP? 

e.  Developing  assigned  CCP  monitoring  plan? 

f .  Developing  corrective  action  approaches? 

g.  Developing  record-keeping  systems? 

h.  Developing  verification  procedures? 
L  Developing  a  generic  HACCP  model? 


Not 
Useful 


Not  At  All 
Useful 


5/13% 

20/50% 

11/28% 

3/8% 

1/3% 

4/11% 

15/41% 

11/30% 

6/16% 

1/3% 

4/10% 

A  V  ew 

26/65% 

7/18% 

3/8% 

0/0% 

5/12% 

22/54% 

10/24% 

4/10% 

0/0% 

5/13% 

22/55% 

11/28% 

2/5% 

0/0% 

4/10% 

23/58% 

10/25% 

3/8% 

0/0% 

4/10% 

22/55% 

9/23% 

5/13% 

0/0% 

3/8% 

21/58% 

9/25% 

3/8% 

0/0% 

6/15% 

24/60% 

7/18% 

3/8% 

0/0% 

WORKSHOP  EXPECTATIONS 

9.    Did  the  workshop  accomplish  what  you  expected?  Yes:  28/72%  No:  11/28% 

Please  explain. 

(23  comments) 

(4)  Yes.  A  generic  model  was  developed  with  some  reserve  on  the  part  of  industry.  The  best  plan 

eventually  was  developed,  however  it  seemed  at  times  some  individuals  were  interested  in  protecting 
their  own  interests.  The  diversity  of  the  group  provided  some  varying  view  points  and  some  hidden 
agendas.  The  workshop  moved  slowly  at  times  but  overall  the  objectives  of  the  agenda  were 
accomplished. 

(4)  No.  Participants  did  not  understand  HACCP  or  its  principles  and  definitions.  Also  tried  to  ignore  hot 
items.  We  did  not  address  the  hard  problems.  CCPs  were  very  watered  down.  The  model  was  too 
generic  and  non-specific 

(4)  No.  Too  many  people  with  too  many  opinions  with  too  little  focus.  I  did  not  expect  a  finished  or  even 
a  generic  model  from  this  large  a  working  group.  There  were  too  many  different  points  of  reference 
going  into  the  workshop. 

(3)  No.  A  tremendous  opportunity  was  wasted.  The  USDA  seemed  to  have  a  plan  in  mind  which  they 
tried  to  impose  on  industry.  When  Control  Points  were  mentioned  by  several  industrial  participants 
they  were  ignored  by  USD  A  representatives.  Questions  were  asked  to  clarify  and  help  bring  all  . 
participants  to  the  same  level  but  they  went  unanswered.  For  example,  it  is  absolutely  essential  to 
finally  define  a  CCP —  do  we  hold  product  when  it  is  out-of-limits?  Once  these  issues  were  ignored 
the  rest  of  the  workshop  went  downhill. 
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(2)   Yes.  The  foundation  for  a  generic  plan  was  developed  The  workshop  was  effective  in  reaching  a 
consensus  behind  which  industry,  government  and  consumers  can  work  together  toward  a  common 
food  safety  goal. 

(2)   Yes.  Excellent  information  exchange.  Good  discussion,  consensus  and  final  decisions. 

(2)   Yes.  However,  I'm  not  sure  HACCP  will  protect  the  consumer  any  more  than  we  are  already  doing. 
I  felt  as  if  we  did  nothing  more  than  rewrite  and  combine  such  established  programs  as  NELS  and 
TQC  We  may  have  gone  overboard  on  CCP  selection. 

(2)  No.  The  resulting  HACCP  model  has  too  many  CCPs.  It  will  be  unmanageable.  The  amount  of 
documentation  required  is  overwhelming  for  most  operations.  I  assumed  the  workshop  would 
develop  a  workable  HACCP  program. 


10.  Did  the  individual  sessions  accomplish  what  you     Yes  for  Yes  for  No  for 

expected?  Please  explain.  All:     17/43%         Some  22/55%       All:  1/3% 

(18  comments) 

(4)  Yes.  Very  useful,  good  idea.  The  workshop  gives  everyone  an  opportunity  to  discuss  in  a 

non-argumental  situation.  Good  place  to  work  and  get  to  know  the  group.  Good  open  discussion 
and  knowledge  exchange. 

(3)  Yes  for  Some.  Marginally.  Many  people  apparently  were  exposed  to  HACCP  for  the  first  time  and 
were  poorly  prepared.  Some  sessions  were  a  waste  of  time  (especially  early  on). 

(3)  Some.  Questions  concerning  the  importance  of  the  CCPs  voted  by  the  group  were  raised.  This  helped 
simplify  the  original  plan.  Perhaps  some  issues  have  become  too  generic.  Chemical  CCP  was  not 
addressed  properly. 

(2)  Some.  Presentations  were  excellent  As  an  introduction  to  HACCP  the  session  was  good,  but  the 
overall  result  of  the  workshop  is  too  complex  to  be  workable. 

(2)  Some.  Several  of  the  mediators  did  not  do  a  good  job  in  controlling  the  sessions.  Information  could 
have  gone  further  in  clearly  establishing  the  objectives)  for  the  moment  and  clearly  establishing  limits 
or  absence  thereof. 

(2)  Some.  Consensus  was  hard  to  achieve.  Our  breakout  group  was  somewhat  biased  toward  using 
government  regulations  as  a  reference.  It  seemed  as  if  the  company  (industry)  could  not  make  a 
decision. 

(1)   Yes.  However,  USDA  intervention  was  greater  than  necessary  and  at  times  impeded  progress. 

(1)  No.  Small  group  sessions  did  not  allow  retracing  of  steps;  also,  because  these  groups  were  assigned 
certain  CCPs  to  work  on,  the  overall  efficacy  of  participant  input  was  severely  diminished.  For 
example,  if  I  had  something  to  say  about  scalding  but  my  particular  small  group  was  not  addressing 
this  CCP,  I  could  not  give  input  on  this  matter  except  in  the  large  group  which  was  somewhat 
intimidating  to  free  exchange. 
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COMMENTS 


11.  Methods  other  than  a  workshop  format  could  have  been  employed  to  develop  a  generic  HACCP  model. 
Examples  of  options  include  an  FSIS  request  for  (a)  comments  on  an  FSIS-generated  model  published  in 
the  Federal  Register,  (b)  development  of  a  model  by  selected  academicians;  or  (c)  submission  to  FSIS  of 
independently-generated  models.  Please  indicate  your  preference  for  one  of  the  above-listed  options, 
or  describe  any  others  not  mentioned  that  you  would  like  to  be  considered  by  FSIS  in  the  future. 

(33  comments) 

Concerning  use  of  a  workshop  format: 

(10)  Stay  with  present  format  No  other  method  should  be  employed.  The  workshop  was  the  best 
approach —  others  mentioned  are  unacceptable.  This  open  forum  was  useful  and  educational  to 
many  of  the  participants.  The  workshop  was  most  efficient  for  agreement  Cooperation  of  industry 
and  USD  A  is  the  preferable  format  Workshops  are  better  than  any  of  the  above  due  to  the  fact  that 
the  people  who  are  going  to  use  the  program  actually  help  set  it  up. 

(3)  I  believe  the  best  way  is  for  a  small  group  at  a  workshop  Gess  than  20  people)  to  review  HACCP 
programs  independently  generated  that  have  already  been  implemented  successfully.  Workshop 
format  is  best;  additional  independently-generated  models  to  use  for  reference  may  have  been 
useful  Have  a  Steering  Committee  develop  a  model  then  present  it  at  a  workshop  for  completion. 

(1)  This  workshop  had  too  many  people  to  come  to  a  consensus;  that  is  why  there  are  so  many  CCPs. 

(1)  The  format  we  had  was  a  good  one;  however,  the  actual  handling  suffered.  Outsiders — turkey 
processors,  FSIS  officials,  and  academicians —  should  not  have  been  allowed  as  participants. 

Concerning  other  options! 

(7)  Option  (O,  submission  to  FSIS  of  independently-generated  models.  Submission  of  models  from 
independent  groups  that  have  experience  with  the  poultry  industry. 

(3)  Develop  model  with  joint  National  Broiler  Council  (NBC),  National  Turkey  Federation  (NTF),  and 
USD  A  representatives  Send  out  questionnaires  and  let  NTF  and  NBC  representatives  develop  model. 
Submission  to  FSIS  of  models  generated  independently  through  a  national  trade  organization. 

(3)  I  do  not  feel  that  anyone,  other  than  industry  professionals,  should  put  programs  together  to  control 
or  regulate  the  industry.  The  HACCP  program  needs  to  be  developed  by  industry  people.  These 
representatives  should  have  knowledge  about  HACCP.  Input  from  industry  is  the  most  logical  way 
to  go. 

(3)  Option  (A)  is  the  best  option  as  it  will  lend  itself  to  the  broadest  array  of  comments  and  input  ideas. 

(1 )  Combination  of  options  (A)  and  (C),  Le,,  submission  to  FSIS  of  independently-generated  models 
combined  with  an  FSIS-generated  model  to  use  as  a  starting  point 

(1)  I  think  a  more  effective  long-term  and  logical  sequence  would  have  been  to  preliminarily  accept  a 
model  by  industry;  test  it;  then  bring  it  back  to  develop  the  hard  rule.  With  the  current  sequence — 
of  the  model  escaping  us  without  a  review  of  the  results —  industry  is  faced  too  harshly  with  the 
possibility  of  losing  input  into  the  final  model. 


12.  What  was  most  beneficial  to  you  about  the  workshop? 
(35  comments) 

(18)  Chance  to  discuss  and  generate  various  options.  Meeting  and  talking  with  fellow  members  of  the 
workshop.  Other  processors7  views.  Open  discussion.  Getting  a  feel  for  general  opinion  of  industry 
towards  HACCP.  Working  with  peers.  Understanding  others7  operations.  Great  to  see  people 
interested  in  implementing  HACCP.  Dialogue.  There  was  a  great  diversity  of  ideas  though  most  of 
this  was  based  on  interpretative  differences.  The  group  as  a  whole  appeared  very  interested  in  food 
safety.  This  workshop  provided  me  a  view  of  what  the  rest  of  the  industry  is  doing  and  the  direction 
of  HACCP.  Chance  to  interface  with  others  and  hear  how  they  are  approaching  this  matter.  Good 
interaction  with  other  companies.  Interfacing  with  other  industry  personneL  Interaction  with 
regulatory  and  industry  management  to  review  current  events  and  future  developments.  I  enjoyed 
meeting  the  experts  of  chicken  processing.  Meeting  key  personnel  for  future  questions.  Group 
participation  and  interaction.  Open  forum. 

(7)  Small  group  activities.  Break-out  groups.  We  made  more  progress  in  small  groups  than  we  did  in 
the  general  session.  In  breakout  groups  we  were  able  to  discuss  issues,  get  a  greater  degree  of 
clarification  where  needed,  and  network  with  other  industry  personnel. 

(6)  Working  through  the  model  The  explanation  of  HACCP  and  the  process  of  developing  the  HACCP 
model.  It  was  a  great  learning  experience  that  will  be  most  useful  in  developing  a  HACCP  plan. 
I  learned  a  lot  about  total  industry  needs  and  concerns.  The  workshop  experience  widened  my 
perspective. 

(2)  All  time  was  efficiently  used.  Enjoyed  the  entire  session! 
(1)  The  computer  system  worked  very  well. 
(1)  The  workbook  materials. 


13.  What  was  least  beneficial  to  you  about  the  workshop? 
(27  conutients) 

(4)  We  were  bogged  down  in  differences  in  semantics.  People  who  were  involved  in  earlier  workshops 
tended  to  take  over  the  workshop.  Too  much  argument  and  debate  over  details  in  the  HACCP 
program.  Extended  discussion  on  some  topics.  Wrangling  over  trivial  issues  that  were  no  more  than 
wording  issues. 

(4)  The  first  day — especially  the  morning.  The  first  day's  agenda  could  have  been  shortened  to  allow 
more  time  to  develop  the  CCP  categories,  then  Corrective  Actions  and  CCP  Descriptions  could  have 
been  better  developed. 

(3)  I  wonder  why  we  need  Subject  Matter  Experts  (USDA) —  the  room  was  filled  with  Subject  Matter 
Experts  (Industry).  A  few  SMEs  were  a  little  too  active  at  a  few  times.  USDA's  intervention  in  small 
group  breakouts  was  least  beneficial. 

(3)  I  disagreed  with  the  voting  procedures.  Inconsistency  in  the  process.  Please  note  that  consensus  was 
seldom  reached —  majority  rule  was  utilized. 

(2)  History  of  poultry  slaughter.  History  of  inspection,  however  presentation  was  excellent. 


(2)   Going  through  the  National  Academy  Plan  and  the  first  day's  agenda  regarding  examples  of  other 
models,  etc  It  was  a  total  waste  of  time  reviewing  the  National  Advisory  Committee's  HACCP  Plan. 

(2)   The  model  presented.  The  HACCP  document  generated. 

(1)   Excessive  pressure  to  implement  a  program  which  significantly  relied  upon  microbiological  testing 
and  reporting.  This  pressure  is  strictly  a  result  of  political  efforts  to  reduce  media  criticism. 

(1)   The  FSIS'  inflexibility —  all  along,  we  were  told  that  we  could  revisit  something  later.  And  then, 
when  asked,  we  were  told  we  had  to  move  forward  Ideas  must  be  revisited 

(1)  Principled. 

(1)  Having  turkeys  and  broilers  mixed. 

(1)  Lack  of  scientific  input 

(1)  Small  working  groups. 

(1)  Large  group  meetings. 


Considering  the  agenda  for  the  first  morning  of  the  workshop,  should  any  of  the  introductory  activities  or 
presentations  be  expanded,  shortened,  or  modified  as  to  content,  to  better  meet  your  needs?  Please  explain. 

(28  continents) 

(11)  Shorten  or  eliminate  the  first  morning's  agenda.  Most  of  the  items  were  a  waste  of  time.  Need  to 
eliminate  about  half  of  first  day  and  get  into  the  development  of  the  program.  Shorten  the  time  spent 
on  definitions  and  objectives.    Shorten  somewhat  but  not  really  necessary.  It  was  nice  but  a  very 
costly  use  of  time.  The  half  day  spent  on  historical  review  and  other  topics  was  redundant  and  not 
germaine  to  the  workshop  objectives.  Too  many  speeches.  Let's  get  political  speeches  out  of  the  way. 
I've  heard  the  same  praises  of  HACCP  several  times  before.  Most  attendees  would  already  know  the 
basics  of  HACCP  and  why  they  are  here. 

(9)  Excellent  as  is.  Good  job.  No  changes. 

(4)  We  spent  too  much  time  explaining  who  people  were  and  what  we  were  to  do.  Needed  more  small 
breakout  group  time.  This  is  a  big  job.  Time  is  critical  in  doing  justice  to  the  core  goal  of  generating 
the  breakout  items  for  CCPs.  Shorten  the  first  morning's  sessions  to  provide  more  time  to  spend  on 
development  of  the  HACCP  program. 

(2)  The  first  hour  should  be  a  review  of  HACCP  principles,  using  the  NRC-MCF  (National  Research 
Council  -  Microbiological  Criteria  for  Food  and  Food  Ingredients)  document  Basis  of  a  HACCP 
Program  should  have  been  explained,  Le.,  CCPs,  Hazard  Analysis. 

(2)  Decrease  introduction  period  for  individuals  and  increase  introduction  period  for  printed  materials. 
Much  of  the  data  and  background  materials  were  new  to  us  and  more  time  for  their  presentation 
would  have  been  helpful.  If  not  possible,  send  out  these  documents  prior  to  the  workshop. 


15.  Please  use  the  rest  of  this  page  to  make  any  additional  comments  that  may  help  us  understand  your 
thoughts  about  the  workshop  and  improve  future  activities. 

(29  comments) 
Workshop-related: 

(5)   Very  impressed  by  the  efforts  put  forth  by  USDA  to  conduct  this  workshop.  Keep  up  the  excellent 
work!  One  of  the  finest  most  educational  and  professionally  conducted  workshops  ever.  Thanks  to 
the  whole  team.  Timelines  and  professional  approach  to  the  whole  process  by  USDA  was  excellent. 
This  process  should  be  supported  by  follow  through  to  ensure  that  pilot  plant  studies  are  conducted. 
No  "political  footballing"  with  this  program.  The  USDA  is  to  be  commended  for  this  effort!  Don't  let 
this  bog  down  in  the  system. 

(4)  Stronger  focus  needs  to  be  exercised  by  facilitators.  Changing  the  definition  of  a  hazard  was  very 
confusing.  USDA  continues  to  ignore  Control  Points  as  extremely  useful  tools.  Not  every  Control 
Point  is  a  Critical  Control  Point  Also,  many  people  became  very  concerned  with  the  concept  of 
a  point  when  discussing  CCPs.  Mv  impression  is  that  a  CCP  can  control  a  system.  During  this 
conference  the  definition  of  Critical  Control  Point  was  changed.  Now  no  one  is  sure  what  is  a 
Critical  Control  Point  As  the  program  is  written  many  of  the  Critical  Control  Points  should  be 
Control  Points. 

(4)  This  was  supposed  to  be  an  industry  program.  When  USDA  participants  were  allowed  to  vote  this 
precept  was  lost  The  packaging  CCP  on  room  temp,  was  greatly  swayed  by  USDA  votes.  The  USDA 
involvement  in  this  program  should  be  decreased,  \Jt.,  no  voting  in  large  group  discussions.  During 
the  last  session  USDA  took  an  unwarranted  and  unwanted  position  of  intervening  into  both  breakout 
group  and  general  group  sessions.  One  of  the  facilitators  imparted  personal  opinions  too  much  on  the 
group  and  proceeded  more  on  personal  judgement  rather  than  according  to  group  consensus. 

(3)  There  was  much  confusion  over  how  set  in  stone  mis  model  is;  made  people  unwilling  to  put  in 
unproven  ideas,  for  fear  that  it  would  become  mandated.  Should  have  been  clearly  stated  that  this 
ultimately  will  be  mandated.  Therefore,  we  should  be  very  careful  not  to  include  suggested  methods, 
etc.  that  will  not  be  productive  in  efficiently  achieving  goal.  Implementation  is  the  key.  Abetter 
vision  of  how  implementation  will  work  in  terms  of  making  a  flexible,  effective  program  should  be 
addressed  at  the  beginning  of  the  conference.  This  partial  omission  restricted  the  commitment  level 
of  interested  participants. 

(2)  I'm  still  not  certain  that  the  turkey  industry  shouldn't  be  in  a  separate  workshop.  Why  were  turkey 
processors  included  as  participants?  They  should  be  observers.  Turkey  should  be  separate  from 
chicken. 

(1)  Include  float  time  in  the  schedule,  ue.,  schedule  for  Friday  8-12  and  use  if  necessary. 

(1)  The  documents  used  to  develop  product  category  were  from  previous  workshops  and  were  not 
appropriate.  Documents  should  be  reviewed  for  appropriateness  prior  to  each  workshop.  The 
group  had  no  leeway  in  deviating  from  the  NAC-MCF  document;  it  is  not  perfect  and  future 
workshops  should  allow  for  criticism  of  any  documents  used. 

(1)  I  think  the  entire  program,  CCP  identification  through  verification,  would  have  been  put  together 
better  if  done  in  small  groups  from  the  outset. 
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HACCP-related: 


(2)   The  industry  should  not  and  USD  A  should  not  consider  themselves  guilty  of  providing  unsafe  food 
products  to  the  consuming  public  Both  should  be  more  aggressive  in  alerting  consumers  of  advances 
made  in  the  last  10, 20, 30  or  more  years  and  how  to  properly  prepare  food.  Industry  and  regulatory 
officials  should  work  together,  however,  I  believe  that  we  have  not  attacked  the  real  problem  Look  at 
the  record:  the  overwhelming  majority  of  food  illnesses  are  caused  by  factors  out  of  industry's  control 
And  yet,  government  chose  to  attack  the  lesser  cause.  Resources  should  be  devoted  to  education,  not 
increased  regulation. 

Other  ideas: 


(2)  The  Puerto  Rico  study  results  would  have  provided  extremely  valuable  information  for  accessing 
and  confirming  CCPs.  Information  on  new  technologies  was  left  out  USD  A /FS  IS  has  done  a  study 
in  Puerto  Rico  but  seemed  more  interested  in  refereed  publications  than  promptly  presenting  this 
vital  material  at  a  critical  seminar  where  these  new  technologies  could  be  used. 

(1 )  Future  workshops  should  try  to  attract  company  personnel  who  will  actually  be  implementing  the 
proposed  plan. 

(1)  Highlight  The  quick  availability  of  hard  copies  was  outstanding;  the  best  I  have  experienced. 
(1)  Not  enough  table  room  in  general  session. 

(1)  It  has  been  very  interesting.  I  hope  my  contributions  were  of  help.  I  look  forward  to  the  development 
of  this  as  a  means  of  self-improvement  for  our  industry. 
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SECTION  II—  OBSERVERS 


SUMMARY  OF  OBSERVERS'  RESPONSES 
EVALUATION  QUESTIONNAIRE 


Workshop  for  HACCP  Model  Development  for 
Poultry  Slaughter,  August  27  -  29, 1991 


SUMMARY   OF   OBSERVERS'  RESPONSES 

RESPONDENT  CHARACTERISTICS 

1. 

Which  grouping  below  best  characterizes  your  current  affiliation? 

Meat  &  Poultry  Industry:    21/72%                      Other  Federal  Government: 
Trade  Association:             1/3%                         Other  Government: 

1/3% 
1/3% 

Educational  Institution:      1/3%  Other: 

3/10% 

News  Media:  1/3% 

2. 

If  you  marked  "Meat  &  Poultry  Industry"  in  Question  1,  please  indicate  all  groupings  below  which  best 
characterize  your  industry  type  and  the  total  production  volume  of  your  establishment(s).  [Low  Volume 
Establishment  =  1  -  59,999  lbs.  produced /Quarter;  Medium  Volume  Establishment  =  60,000  - 1,000,000  lbs. 
produced /Quarter;  High  Volume  Establishment  =  more  than  1,000,001  lbs.  produced /Quarter] 

Meat  &  Poultry  Industry                Low  Volume  Establishment(s): 

Medium  Volume  Establishment(s): 
High  Volume  Establishment(s): 
Unspecified  Volume: 

0/0% 
0/0% 
7/35% 
3/15% 

Poultry  Industry                          Low  Volume  Establishment(s): 

Medium  Volume  Establishment(s): 
High  Volume  Establishment(s): 

0/0% 
0/0% 
10/50% 

3. 

Days  attended  (mark  all  that  apply):           1  Day:  0/0%              2  Days:  1/3% 

3  Days: 

28/97% 

4. 

Please  check  one:                                  Observer:    29           Participant:  0 

HACCP  MODEL  DEVELOPMENT 

5. 

Very                          Somewhat  Not  Very 
Successful  Successful  Successful  Successful  Unsuccessful 

Overall,  how  successful  was  the  workshop  in: 

Don't 
Know 

a.  Identifying  Critical  Control  Points  (CCPs)?  1/3%       7/24%      17/59%  1/3% 

3/10% 

0/0% 

b.  Deterrriining  critical  limits  for  each  CCP?     0/0%      .  5/18%      18/64%  4/14% 

0/0% 

1/4% 

c.  Developing  monitoring  plan  activity  for      0/0%        7/26%      17/63%  2/7% 
identified  CCPs? 

0/0% 

1/4% 

d.  Developing  corrective  action  approaches?    0/0%       9/33%      13/48%  3/11% 

0/0% 

2/7% 

e.  Developing  record-keeping  systems?         0/0%       8/32%      12/48%  3/12% 

0/0% 

2/8% 

f.  Developing  verification  procedures?          0/0%       8/32%      13/52%  2/8% 

0/0% 

2/8% 

g.  Developing  a  generic  HACCP  model?        0/0%       8/31%      15/58%  1/4% 

1/4% 

1/4% 

NOTE:  Results  are  presented  as  number  of  responses/%  of  responses  for  each  category. 
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DESIGN  AND  OPERATION  OF  WORKSHOP 

6. 

Please  rate  the  workshop  in  terms  of: 

More  than 
Adequate 

Adequate 

Marginally 
Adequate 

Inadequate 

Very 

Inadequate 

Don't 
Know 

a. 

Development  of  a  generic  product /process 
description 

1  11  01 

t  Q/CCOl 

1  y/ oo  to 

olio  /c 

1  /I  01 

115  to 

U/U% 

0/0% 

b. 

Development  of  process  flow  diagram 

3/10% 

23/79% 

2/7% 

1/3% 

0/0% 

0/0% 

c. 

Organization  of  the  three  days 

5/17% 

19/66% 

3/10% 

2/7% 

0/0% 

0/0% 

d. 

Use  of  small  "break-out"  groups 

C  /1  Q  CI 
5/1 O  to 

lb/57 /c 

A  /"t  A  CI 

4/14% 

1  1 A  OI 

1/4% 

0/0% 

2/7% 

e. 

Reporting  of  small  "break-out"  group 
activities  to  entire  group 

O  /TO/ 
JJ  /  to 

*>n/7i  oi 
Vol  / 1  to 

4/  14  10 

1  /A  07 

1/4% 

U/Uvc 

1/4% 

f. 

Opportunity  for  input  from  Observers 

0/0% 

10/35% 

7/24% 

8/28% 

4/14% 

0/0% 

Oonortunitv  for  inout  from  Particioants 

n  fyt  oi 

lb/  3D  10 

lit  no? 
3/1  U% 

U/U% 

1/3% 

0/0% 

h. 

Effectiveness  of  facilitators  in  helping 
achieve  workshop  objectives 

3/10% 

17/59% 

5/17% 

0/0% 

3/10% 

1/3% 

i. 

Effectiveness  of  subject  matter  experts  in 
helping  achieve  workshop  objectives 

1/3% 

13/45% 

5/17% 

7/24% 

2/7% 

1/3% 

j- 

Effectiveness  of  group  leaders  in  helping 
achieve  workshop  objectives 

0/0% 

20/71% 

5/18% 

0/0% 

1/4% 

2/7% 

k. 

Effectiveness  of  group  reporters  in  helping 
achieve  workshop  objectives 

0/0% 

21/75% 

4714% 

0/0% 

0/0% 

3/11% 

1. 

Completion  of  workshop  objectives. 

0/0% 

12/41% 

12/41% 

3/10% 

0/0% 

2/7% 

7. 

Please  indicate  your  degree  of  agreement 
with  each  of  the  following  statements. 

Strongly 
Agree 

Agree 

Neutral 

Disagree 

Strongly 
Disagree 

Don't 
Know 

a. 

The  HACCP  model  developed  by  the 
workshop  participants  is  a  viable  frame- 
work for  the  industry  to  implement 
on  a  test  basis. 

0/0% 

9/32% 

8/29% 

9/32% 

1/4% 

1/4% 

b. 

The  workshop  format  offers  a  sound 
approach  for  producing  a  generic  HACCP 
model  for  other  meat  and  poultry  products 

1/4% 

14/50% 

6/21% 

5/18% 

1/4% 

1/4% 

c. 

The  presence  of  Observers  was  beneficial 
to  the  workshop. 

2/7% 

19/66% 

6/21% 

2/7% 

0/0% 

0/0% 

d. 

The  workshop  resulted  in  a  useful  generic 
HACCP  model  developed  by  industry 
participants. 

0/0% 

8/28% 

8/28% 

9/31% 

2/7% 

2/7% 
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WORKSHOP  MATERIALS  (Observers:  Please  skip  Question  8  and  continue  with  Question  9.) 

Very  Somewhat     Not  Not  At  All 

Useful  Useful         Useful  Useful  Useful 

8.  Overall,  how  useful  were  the  written  materials  for: 

a.  Assessing  finished  product  hazards? 

b.  Assessing  ingredient  hazards? 

c.  Determining  placement  of  CCPs  on  flow 
diagram? 

d.  Determining  critical  limits  for  each  CCP?  OBSERVERS  WERE  INSTRUCTED  TO 

e.  Developing  assigned  CCP  monitoring  plan?  SKIP  THIS  QUESTION. 

f .  Developing  corrective  action  approaches? 

g.  Developing  record-keeping  systems? 

h.  Developing  verification  procedures? 

i.  Developing  a  generic  HACCP  model? 

WORKSHOP  EXPECTATIONS 

9.  Did  the  workshop  accomplish  what  you  expected?  Yes:  17/65%  No:  9/35% 
Please  explain. 

(20  comments) 

(5)    Yes.  The  workshop  was  different  from  what  I  expected  but  I  was  pleased  with  the  workshop  and  its 
objectives  once  I  found  out  what  they  were.  I  am  not  sure  a  solid  HACCP  program  could  be 
developed  in  only  3  days  but  I  believe  enough  was  done  to  form  a  foundation.  The  workshop  was 
very  useful  for  training  individuals  about  how  to  develop  a  HACCP  plan.  The  group  basically  headed 
in  the  right  direction  and  accomplished  about  as  much  as  possible  for  a  large  group  in  a  very  short 
period  of  time.  I  found  the  process  of  designing  the  generic  model  very  helpful  as  well  as  the  discus- 
sion surroundingit.  It  identified  problem  areas,  established  type  exposure,  and  set-up  monitoring  pro- 
cedures. 

(3)   No.  The  plan  developed  is  more  of  a  quick  fix  for  known  weaknesses  of  the  slaughter  procedures  than 
a  plan  of  preventive  measures  ensuring  a  wholesome  product.  Many  potential  hazards  were  not 
considered  or  were  voted  out  of  the  plan  based  on  the  feelings  of  the  participants  and  not  on  hard 
data.  The  purpose  of  this  workshop  was  to  develop  a  generic  plan  that  could  then  be  tested  in  a  pilot 
program  to  determine  the  efficacy  of  the  plan.  If  potential  hazards  are  eliminated  prior  to  this  test 
program,  then  we  can't  be  sure  that  the  HACCP  plan  is  truly  effective.  Not  enough  facts  presented 
or  pressed  for  on  microbe  counts,  results  of  testing.  Everything  was  done  by  voting  based  on  opinion. 

(2)   Yes.  I  expected  many  CCPs  and  that's  what  happened.  Concept  of  Control  Points  (CP)  rather  than 
CCPs  needs  to  be  seriously  considered.  Sixteen  different  CCPs  is  unworkable  from  documentation 
required  standpoint  and  plant  implementation,  monitoring,  verification,  etc.  Definition  of  a  CCP 
needs  to  be  changed /modified  to  establish  what  is  truly  a  health  hazard  or  risk.  Basically,  needed 
more  hands-on  type  of  people  and  fewer  desk  jockeys  as  participants.  Some  of  these  people  wanted 
to  craft  the  plan  in  confusing  language  and  would  probably  argue  over  the  definition  of  a,  an,  or  the. 

(2)    Yes.  I  received  adequate  information  to  discuss  the  objectives  of  HACCP.  I  learned  a  lot  about  the 
industry. 
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(2)    No.  Too  many  of  the  monitoring  steps  are  USDA  regulations.  What  did  we  accomplish?  Issues  are 
still  unclear.  Most  issues  addressed  arc  covered  within  the  NELS  system  now. 

(2)    No.  The  basic  concept  of  HACCP  was  not  followed  in  any  phase  of  the  workshop.  Would  suggest  that 
all  people  involved  be  required  to  attend  a  HACCP  school  prior  to  developing  any  model.  HACCP 
has  to  be  a  manageable  program.  It  is  not  effective  otherwise.  When  there  are  more  CCPs  than  CPs 
it  becomes  too  complicated  to  manage  effectively.  The  workshop  lost  sight  of  what  HACCP  really  is. 

(1)    Yes.  My  expectations  were  low. 

(1)    Yes.  The  use  of  small  group  discussions  was  especially  beneficial.  Use  small  groups  more  to  develop 
and  bring  items  up  for  discussion.  As  the  workshop  progressed,  I  think  more  and  more  people 
realized  what  a  CCP  was  and  saw  how  it  was  defined  and  realized  the  implication  of  CCPs.  This 
then  allowed  industry  representatives  to  begin  to  eliminate  CCPs  that  could  not  be  controlled 
(personal  hygiene,  etc.). 

(1)  No.  Although  such  a  workshop  is  a  fair  method  to  represent  industry,  there  were  too  many  expert 
participants.  In  the  best  interests  of  time  management,  perhaps  the  selection  of  participants  could 
come  from  Area  workshops  held  in  preparation  for  a  national  workshop. 

(1)  No.  I  expected  more  input  from  FSIS  as  to  what  would  be  required  of  plants.  Their  judgement  will  be 
the  final  decision. 

10.   Did  the  individual  sessions  accomplish  what  you      Yes  for  Yes  for  No  for 

expected?  Please  explain.  All:     6/24%  Some:  17/68%        All:  2/8% 

(12  comments) 

(2)  Some.  The  small  breakout  groups  allowed  for  an  easier  exchange  of  ideas.  Smaller  groups  were 
efficient,  however,  the  principle  of  HACCP  was  not  followed. 

(2)    Some.  I  cannot  comment  on  the  breakouts  because  I  did  not  attend  these  sessions,  but  the  general 
session  helped  me  understand  HACCP  and  plan  development.  Some  sessions  were  not  as  smoothly 
organized  as  they  could  have  been.  More  clearly  explained  details  of  what  should  be  accomplished  in 
each  session  would  be  beneficial,  but  overall,  good  organization  for  a  difficult  task. 

(1)    Yes.  I  did  expect  more  input  from  participants  who  did  not  voice  their  opinions  or  ideas  in  the  large 
groups.  More  insight  and  ideas  were  therefore  examined  in  the  small  group  sessions. 

(1)   Yes.  The  brainstorming  was  interesting  and  so  was  seeing  it  all  narrowed  down  to  one  statement. 

(1)   Some.  It  appeared  that  the  group  kept  wandering  outside  the  scope  of  the  agenda  to  develop  a 

HACCP  program.  Group  was  "hung  up"  on  the  exception  rather  than  the  general  rule.  Perhaps  the 
definitions  of  the  items  in  question  should  be  shown  on-screen  to  help  keep  people  from  straying 
outside  the  scope  (i.e.,  hazards,  CCPs,  etc.).  Definitions  including  the  terms  likely  to  exist  and 
does  not  exist  possibly  would  be  more  applicable  if  the  wording  were  changed  to  may  exist  but  not  likely. 

(1)   Some.  It  was  difficult  in  the  beginning  as  an  observer  to  follow  along  because  printed  materials  were 
not  made  available.  This  situation  improved  as  notebooks  and  printed  materials  were  provided  to 
observers. 

(1)   Some.  Learned  from  conversations  at  breaks,  lunches,  efc. 
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(1)    Some.  The  initial  sessions  did  not  sufficiently  define  the  degree  of  potential  hazards  that  should  be 
considered.  Hard  data  are  needed  to  judge  whether  or  not  a  potential  hazard  is  important.  Hazard 
potential  cannot  be  decided  by  gut  feel  or  memory;  get  the  facts  before  eliminating  a  hazard  from  the 
HACCP,  otherwise  err  on  the  side  of  safety. 

(1)    No.  I  expected  observers  to  receive  more  materials  to  help  us  follow  proceedings. 

(1)    No.  The  workshop  objectives  could  have  been  developed  by  the  group  as  a  whole  rather  than  using 
the  small  breakout  group  format. 

COMMENTS 

11.   Methods  other  than  a  workshop  format  could  have  been  employed  to  develop  a  generic  HACCP  model. 
Examples  of  options  include  an  FSIS  request  for  (a)  comments  on  an  FSIS-generated  model  published  in 
the  Federal  Register;  (b)  development  of  a  model  by  selected  academicians;  or  (c)  submission  to  FSIS  of 
independently-generated  models.  Please  indicate  your  preference  for  one  of  the  above-listed  options, 
or  describe  any  others  not  mentioned  that  you  would  like  to  be  considered  by  FSIS  in  the  future. 

(24  comments) 

Concerning  use  of  a  workshop  format: 

(6)   Workshop  is  the  best  method.  At  least  people  talk  to  each  other.  Good  workshop.  The  workshop 
format  is  effective. 

(1)   This  method  with  industry  participants  is  probably  the  best  method  to  develop  a  working  model. 
However,  need  to  realize  that  not  everything  is  a  CCP —  there  must  be  control  points  that  are 
controlled  by  industry.  Successful  HACCP  programs  combine  CCPs  and  CPs.  Successful  HACCP 
programs  are  normally  limited  to  approximately  5  CCPs,  otherwise  there  is  no  control  since  the 
program  is  too  cumbersome  and  works  against  itself. 

(1)   I  would  like  to  see  another  Poultry  Slaughter  workshop  completed  with  USDA  inspectors  and  circuit 
supervisors.  Even  though  some  circuit  supervisors  were  present  here,  they  were  overruled  by  in- 
dustry representatives.  It  is  understandable  that  the  inspectors  and  industry  representatives  would 
not  be  able  to  achieve  the  workshop  objectives  in  the  same  workshop;  separate  workshops  would 
produce  model  plans  from  both  sides. 

(1)   I  do  not  believe  a  viable  working  plan  can  ever  be  developed  by  not  including  those  who  must 

implement  the  plan  and  those  who  must  administer.  A  joint  effort  such  as  this  workshop  will  be  the 
best  activity. 

(1)   The  workshop  format  worked  very  well.  Additional  selected  academicians  could  have  offered  a  more 
neutral  forum  instead  of  allowing  multiple  representatives  from  one  company  to  attend.  Multiple 
representatives  from  one  company  gives  unfair  voting  rights  to  those  companies. 

Concerning  other  options: 

(5)   Option  (C),  submission  to  FSIS  of  independently-generated  models,  is  the  best  approach  if  not  using 
a  workshop  format.  (C)  has  merit  but  may  take  too  long  for  FSIS  to  review  and  would  still  have  to 
do  (A)  after  that.  Option  (C)  should  be  utilized  and  better  designed  plans  illustrated  to  the  industry. 

(3)    "Definitely  no"  for  Option  (B).  It  doesn't  allow  for  enough  industry  input. 
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(2)    Development  of  model  by  HACCP  experts  such  as  Don  Corlett.  Independent  models  should  have 
been  submitted  and  discussed.  Everyone  would  have  benefited  by  others'  ideas. 

(2)    "No"  for  Option  (A). 

(1 )    Option  (A),  comments  on  an  FSIS-gencratcd  model  published  in  the  Federal  Register,  would  generate 
the  most  input  in  the  shortest  period  of  time  and  therefore  facilitate  development  at  a  faster  pace. 

(1)    "No"  for  Option  (C) —  too  much  variation  in  programs. 


12.   What  was  most  beneficial  to  you  about  the  workshop? 
(28  comments) 

(8)   Going  through  the  actual  process  of  designing  a  HACCP  model.  I  have  a  better  understanding  of  the 
step-by-step  process  in  developing  a  HACCP  program.  The  hazard  assignment  exercise.  Very  good 
for  training  only.  Seeing  the  plan  develop.  Gaining  a  clearer  understanding  of  what  HACCP 
represents  and  is  being  designed  to  accomplish.  Understanding  of  how  a  HACCP  program  would 
work  in  everyday  situations.  Achieving  a  general  understanding  of  HACCP. 

(5)   Observing  the  various  opinions  voiced  by  industry  and  USDA.  The  interaction  of  the  different 
people  present  and  more  exposure  to  the  reasons  behind  HACCP.  The  opportunities  for  industry, 
government,  and  academia  to  pool  their  respective  knowledge.  Hearing  different  comments  from 
other  companies.  It  made  me  aware  of  how  far  ahead  some  companies  are.  There  are  some 
aggressive  companies  and  hopefully  others  will  follow. 

(5)   Small  breakout  sessions  and  their  reports.  Small  group  discussions  were  especially  good.  More  was 
accomplished  here  than  at  any  other  point  in  the  program.  The  reporting  on  the  assigned  items  and 
discussion  that  ensued  was  better  than  large  group  discussions.  Small  breakout  sessions  and  presence 
of  observers. 

(3)   The  opportunity  to  meet  USDA  and  other  industry  personnel  and  to  develop  a  working  relationship 
with  them.  The  presence  of  USDA  communications  staff  and  their  willingness  to  help  me  develop 
news  stories  and  obtain  the  information  needed. 

(2)   The  discussions —  industry  often  was  conservative  and  USDA  wanted  more  action.  The  FSIS  people 
were  aggressive;  industry  sat  on  its  haunches.  Forcing  industry  to  respond  and  acknowledge  the  need 
for  their  input. 

(2)   All  was  beneficial.  Overall  it  was  interesting. 

(1 )   Being  able  to  observe  the  overall  confusion  and  lack  of  knowledge  concerning  HACCP. 

(1)   The  use  of  the  computer  terminals  was  effective.  The  quick  distribution  of  hard  copies  of  CCP 
information  was  equally  effective. 

(1)   The  screens  were  difficult  to  read,  especially  from  the  far  right  and  left  edges  of  the  observers'  sections. 
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13.    What  was  least  beneficial  to  you  about  the  workshop7 
(IS  comments) 

(5)    Some  people  present  had  no  idea  what  the  basis  of  a  HACCP  plan  is,  let  alone  how  to  go  about 
developing  one.  Too  many  people  wanted  a  specific  HACCP  program  directed  toward  their  own 
particular  needs.  It  is  basically  impossible  for  50  people  to  develop  a  good  generic  HACCP  plan  in 
3  days  for  use  in  a  pilot  program.  If  everyone  here  were  trained  in  HACCP  development  then  they 
might  be  able  to  to  develop  such  a  plan,  but  they  weren't.  The  HACCP  model  developed  was  a  good 
start  but  has  many  holes  and  will  not  be  complete  for  pilot  testing.  The  lack  of  ability  to  properly 
determine  CCPs  because  of  confusion/lack  of  knowledge  concerning  HACCP.  Lack  of  knowledge  of 
some  industry  participants  and  the  obvious  individual  agendas.  Determining  why  HACCP  is 
necessary;  CCPs  seem  to  be  covered  already. 

(4)   The  discussion  tended  to  get  bogged  down  at  times  when  discussion  leaders/facilitators  allowed 

participants  to  rehash  topics  that  already  had  been  voted  on.  Arguing  most  points.  The  instructors  in 
large  group  presentations  were  not  as  effective  as  they  needed  to  be.  The  participants  needed  more 
explanation  of  hazards  and  CCPs.  Also,  one  instructor  repeatedly  lost  control  of  the  class.  At  one 
point  a  participant  stood  in  front  of  the  class  leading  the  instruction.  Facilitators  are  to  follow 
timelines  and  to  solicit  input  but  they  also  need  to  keep  the  focus  by  restating  definitions,  as  needed, 
when  the  discussion  gets  off  track. 

(4)    As  an  observer,  the  breakout  sessions  were  the  least  beneficial.  Not  being  able  to  participate. 
Observers  had  very  little  participation.  Lack  of  material  for  observers. 

(2)   Tuesday  morning  was  lost  by  bearing  the  dead  horse  over  and  over.  The  first  day!  Too  much  ado 
about  nothing. 

(1)   The  practice  of  reaching  a  consensus  was  completely  abandoned.  The  decisions  were  reached  by 
means  of  voting.  At  times  only  28  out  of  approximately  45  participants  voted.  Voting  creates  a 
win/lose  situation  which  should  be  avoided. 

(1)   The  lack  of  a  true  HACCP  authority  to  help  guide  the  workshop  and  provide  explanations  for  HACCP 
terms,  definitions  and  instructions.  In  addition,  an  authority  on  the  current  USDA  regulations  would 
have  been  helpful. 

(1)   Because  this  was  my  first  workshop  it  is  difficult  to  say.  I  guess  it  will  be  the  time  I  took  to  be  here  if 
nothing  positive  comes  from  the  workshop. 


14.  Considering  the  agenda  for  the  first  morning  of  the  workshop,  should  any  of  the  introductory  activities  or 
presentations  be  expanded,  shortened,  or  modified  as  to  content,  to  better  meet  your  needs?  Please  explain. 

(19  comments) 

(10)  The  first  morning's  sessions  were  excellent.  It  was  handled  well.  No  changes.  O.K.  Leave  as  is. 
It  was  adequate. 

(6)   All  should  be  shortened.  Get  on  with  the  program  itself.  There  is  enough  time  to  meet  people 
especially  when  biographical  sketches  of  people  present  were  given  out.  The  entire  workshop 
seemed  to  be  rushed.  More  time  should  be  allotted  for  breakout  team  discussions.  Too  much  time 
on  introductions.  First  morning  could  move  faster.  More  than  1/2  of  the  1st  day  was  wasted  on 
unnecessary  introductory  comments.  This  could  be  significantly  shortened  by  curtailing  speech- 
making  that  does  not  add  to  the  development  process. 
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(2)    just  about  right  but  some  of  the  information  could  have  been  shared  the  night  before.  Possibly  shorten 
the  agenda  some.  Generally  good  overall. 

(1)    Give  experience  and  expertise  of  FSIS  personnel's  time  spent  in  poultry  and /or  meat  plants. 


15.   Please  use  the  rest  of  this  page  to  make  any  additional  comments  that  may  help  us  understand  your 
thoughts  about  the  workshop  and  improve  future  activities. 

(20  comments) 

Workshop-related: 

(4)   Notebooks  and  other  written  materials  need  to  be  provided  to  observers  at  the  beginning  of  the 
workshop.  The  observers  need  to  be  provided  with  tables,  water,  and  the  same  information  as  the 
participants.  Observers  need  to  be  able  to  state  their  views  also,  especially  during  or  after  large  group 
discussions.  Eliminate  1st  day's  observer  comments  and  allow  time  for  this  activity  somewhere  else 
in  the  program  (after  CCPs  or  small  group  exercises). 

(2)   If  HACCP  programs  are  to  be  truly  industry  developed  then  USDA  personnel  should  not  vote  when 
decisions  are  made.  USDA  should  be  advisors  only.  There  were  too  many  FSIS  employees 
participating  in  the  decision-making  steps.  Overall  the  insight  provided  by  USDA  was  very  good 
and  professional,  however,  several  comments  by  individual  FSIS  personnel  exemplified  a  typical 
superiority  attitude  in  relationships  with  plant  personnel.  Comments  like  "That's  your  job,"  were  not 
beneficial  to  the  dialogue  at  the  time  and  placed  participants  in  a  combative  position.  However,  the 
Washington-based  USDA  personnel  were  very  professional. 

(2)   More  input  was  needed  from  USDA  on  regulations  when  questions  regarding  acceptable  limits  arise. 
USDA  should  take  a  more  active  role  in  steering  the  participants  in  the  direction  that  USDA  will 
require. 

(2)   Some  USDA  facilitators  need  to  improve  their  control  in  large  group  discussions.  At  times  discussions 
got  out  of  hand  and  this  is  when  the  facilitators  need  to  mediate  better!  During  the  large  group 
brainstorming  sessions  for  ideas,  I  suggest  you  show  all  suggestions  given  by  the  participants.  Often 
times  the  participants  would  say,  "What  did  we  say  a  minute  ago?"  By  showing  all  the  ideas  for  a 
hazard  or  CCP,  less  confusion  would  result  and  a  true  consensus  could  be  reached.  Without  a  true 
consensus  there  will  not  be  total  commitment  from  the  industry  for  implementing  the  plan. 

(1 )   Would  be  best  to  solicit  comments  on  an  FSIS-generated  model  then  hold  this  type  of  workshop  for 
review.  People  reviewing  (sent  by  companies/by  industry)  should  be  trained  for  HACCP 
development.  Although  I'm  a  member  of  a  large  food  processor,  I  believe  there  is  too  much  room 
for  industry  to  deviate  from  what  should  be  done  to  what  is  convenient  for  industry.  FSIS  and  science 
must  be  adamant  about  certain  CCPs  and  defend  that  position.  Much  of  the  same  discussion  that  we 
had  in  the  three  days  will  still  occur  but  we  would  have  been  a  lot  further  along  and  companies  would 
already  have  had  the  opportunity  to  input  on  a  model  before  the  meeting.  I  know  this  idea  is  a  harder 
sell  than  the  way  FSIS  is  doing  it  now. 

(1)   I  am  not  industry  or  USDA,  but  a  supplier  to  both.  Meeting  the  people  and  seeing  how  the  program 
works,  the  problems  etc.  helps  us  provide  better  goods  and  services  to  both.  I  enjoyed  meeting 
Dr.  Crawford  and  his  staff. 

(1)   I  would  suggest  that  the  set-up  be  done  in  a  round  table  or  horseshoe  arrangement  for  better  eye 
contact  and  clearer  understanding  of  the  program  and  comments. 

(1)    Thank  you. 
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I 

UACCF-rclatcd: 

(2)    Since  our  company  docs  much  more  than  the  HACCP  model,  I  don't  believe  much  was  accomplished 
here.  There  was  research  data  that  was  not  shared  that  would  have  been  helpful.  I  have  the  feeling 
when  it  got  to  really  making  a  commitment  there  is  a  lack  of  trust  in  the  intent  of  FSIS...  Are  we 
(industry)  choosing  which  rope  we  will  be  hung  with?  I  know  several  people  who  feel  the  whole 
exercise  was  a  political  tool  to  allow  USDA  to  say  it  is  a  tool  that  industry  cooperatively  developed 
with  yUSDA  but  it  allows  USDA  to  stay  out  of  the  business  of  microbial  testing. 

(2)    The  USDA  failed  to  use  the  expertise  available  in  the  US  to  aid  in  the  development  of  a  HACCP  model. 
As  a  result  we  will  have  a  program  which  in  no  way  resembles  a  true  HACCP  plan.  There  were  too 
many  CCPs  in  the  HACCP  model.  Keep  CCPs  to  a  minimum  to  keep  a  good  focus! 

(1)   Excellent  discussion,  however  1  question  whether  or  not  the  plan  should  be  used  as  a  template  for  the 
turkey  industry. 

(1)   This  program  needs  to  start  before  the  plant  level.  It  needs  to  start  at  the  hatchery  and  continue 
through  each  level. 
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Summary  of  the 
Evaluation  Results  of  the 
Workshop  for  HACCP 
Model  Development 


Phoenix,  Arizona,  December  3  -  5,  1991 


INTRODUCTION 


This  report  presents  the  evaluation  results  from  the  Food  Safety  and  Inspection  Service  CFSIS) 
of  the  United  States  Department  of  Agriculture  (USDA)  workshop  to  develop  a  generic 
model  for  a  Hazard  Analysis  and  Critical  Control  Point  (HACCP)  plan  for  fresh  ground  beef. 
Representatives  from  industry,  trade  associations  and  government  met  in  Phoenix,  Arizona, 
December  3-5, 1991,  for  this  workshop  which  focused  on  ground  beef  patties.  The  workshop 
was  open  to  the  public  for  observation,  on  a  reserved  space  basis. 

Participants  and  Observers  were  asked  to  complete  a  questionnaire  which  asked  them 
questions  related  to  the  design  and  operation  of  the  workshop,  the  workshop  materials,  the 
process  employed  in  developing  a  generic  HACCP  model,  and  their  views  concerning  the 
workshop  approach  and  any  suggested  improvements.  Of  the  31  Participants  and  12  Observers 
in  attendance,  28  Participants  (90%)  and  6  Observers  (50%)  responded  to  the  questionnaire. 

This  evaluation  activity  is  one  in  a  series  of  on-going  evaluation  tasks  that  are  a  part  of  the 
HACCP  Implementation  Study.  The  results  of  this  evaluation  should  serve  to  help  plan 
subsequent  workshops  and  to  provide  background  information  to  aid  in  future  planning 
decisions  related  to  HACCP. 


REPORT  PLAN 


This  report  is  organized  into  two  sections.  Section  I  presents  the  results  and  summary  of 
Participants'  responses  and  Section  II  presents  those  for  the  Observers.  Results  of  the 
multiple-choice  questions  are  presented  in  a  format  based  on  that  of  the  original  questionnaire. 
The  responses  are  reported  by  number  and  percent  of  responses  for  each  category.  Percentages 
reported  are  based  on  the  number  of  responses  to  each  question  and  do  not  always  total 
100  percent  due  to  arithmetic  rounding. 

Results  of  the  open-ended  questions  are  also  summarized  in  each  Section.  A  content  analysis 
was  performed  for  all  hand-written  comments.  Respondents'  remarks  are  paraphrased 
and  grouped  by  question.  Some  respondents  made  multiple  comments  while  others  made 
none,  thus  the  number  of  comments  per  question  does  not  equal  the  number  of  respondents. 
The  number  of  respondents  who  concurred  with  each  comment  is  listed  parenthetically. 


SECTION  I—  PARTICIPANTS 


SUMMARY  OF  PARTICIPANTS'  RESPONSES  TO 
EVALUATION  QUESTIONNAIRE 


Workshop  for  HACCP  Model  Development  for 
Fresh  Ground  Beef,  December  3  -  5, 1991 


SUMMARY  OF  PARTICIPANTS'  RESPONSES 

RESPONDENT  CHARACTERISTICS 

1. 

Which  grouping  below  best  characterizes  your  current  affiliation? 

Meat  &  Poultry  Industry:    23/82%                     Private  Laboratory: 

1/4% 

Soup  and  Dressing:           1/4%                        Equipment  Manufacturer: 

1/4% 

Trade  Association:            1/4%                        Industry  Consultant: 

1/4% 

2. 

If  you  marked  ''Meat  &  Poultry  Industry7'  in  Question  1,  please  indicate  all  groupings  below  which  best 
characterize  your  industry  type  and  the  total  production  volume  of  your  establishment s).  [Low  Volume 
Establishment  =  1  -  59,999  lbs.  produced /Quarter;  Medium  Volume  Establishment  =  60,000  - 1,000,000  lbs. 
produced /Quarter;  High  Volume  Establishment  =  more  than  1,000,001  lbs.  produced /Quarter] 

Meat  Industry                           Low  Volume  Establishment(s): 

Medium  Volume  Establishments): 
High  Volume  Establishment(s): 

0/0% 

4/17% 

11/48% 

Meat  &  Poultry  Industry               Low  Volume  Establishment(s): 

Medium  Volume  Establishments): 
High  Volume  Establishments): 

0/0% 
1/4% 
7/30% 

3. 

Days  attended  (mark  all  that  apply):          1  Day:  1/4%             2  Days:  1/4% 

3  Days:  26/93% 

4. 

Please  check  one:                                Observer:    0            Participant:  28 

HACCP  MODEL  DEVELOPMENT 

Very                         Somewhat  Not           Very  Don't 
Successful  Successful  Successful  Successful  Unsuccessful  Know 

5. 

Overall,  how  successful  was  the  workshop  in: 

a.  Identifying  Critical  Control  Points  (CCPs)?  6/22%      17/63%     4/15%  0/0% 

0/0%  0/0% 

b.  Determining  critical  limits  for  each  CCP?    6/22%      17/63%     4/15%  0/0% 

0/0%  0/0% 

c.  Developing  monitoring  plan  activity  for     7/28%      13/52%     5/20%  0/0% 
identified  CCPs? 

0/0%  0/0% 

d.  Developing  corrective  action  approaches?   7/28%      12/48%     6/24%  0/0% 

0/0%  0/0% 

e.  Developing  record-keeping  systems?         5/20%      6/24%      13/52%  0/0% 

0/0%  1/4% 

f.  Developing  verification  procedures?          5/20%      9/36%      11/44%  0/0% 

0/0%  0/0% 

g.  Developing  a  generic  HACCP  model?        8/32%      12/48%     5/20%  0/0% 

0/0%  0/0% 

NOTE:  Results  are  presented  as  number  of  responses/%  of  responses  for  each  category. 
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DESIGN  AND  OPERATION  OF  WORKSHOP 


Please  rate  the  workshop  in  terms  of: 

More  than 
Adequate 

Marginally 

Adequate  Inadequate 

Very 
'  Inadequate 

Don't 
Know 

a.  Development  of  a  generic  product /process  9/32% 
description 

16/57% 

2/7% 

1/4% 

0/0% 

0/0% 

b.  Development  of  process  flow  diagram 

7/25% 

16/57% 

2/7% 

3/11% 

0/0% 

0/0% 

c.  Organization  of  the  three  days 

14/52% 

10/37% 

2/7% 

1/4% 

0/0% 

0/0% 

d.  Use  of  small  "break-out"  groups 

17/61% 

9/32% 

2/7% 

0/0% 

0/0% 

0/0% 

e.  Reporting  of  small  "break-our"  group 
activities  to  entire  group 

11/39% 

13/46% 

4/14% 

0/0% 

0/0% 

0/0% 

f .  Opportunity  for  input  from  Observers 

7/30% 

7/30% 

3/13% 

1/4% 

1/4% 

4/17% 

g.  Opportunity  for  input  from  Participants 

21/75% 

0/0% 

0/0% 

a /n  fir 

0/0% 

0/0% 

h.  Effectiveness  of  facilitators  in  helping 
achieve  workshop  objectives 

17/61% 

10/36% 

0/0% 

0/0% 

1/4% 

0/0% 

i.  Effectiveness  of  subject  matter  experts  in 
helping  achieve  workshop  objectives 

8/29% 

15/54% 

5/1  R% 

0/0% 

0/0% 

j.  Effectiveness  of  group  leaders  in  helping 
achieve  workshop  objectives 

12/43% 

14/50% 

2/7% 

0/0% 

0/0% 

0/0% 

k.  Effectiveness  of  group  reporters  in  helping  9/32% 
achieve  workshop  objectives 

16/57% 

3/11% 

0/0% 

0/0% 

0/0% 

1.  Completion  of  workshop  objectives. 

12/43% 

13/46% 

3/11% 

0/0% 

0/0% 

0/0% 

Please  indicate  your  degree  of  agreement 
with  each  of  the  following  statements. 

Strongly 
Agree 

Agree 

Neutral 

Disagree 

Strongly 
Disagree 

Don't 
Know 

a.  The  HACCP  model  developed  by  the 
workshop  participants  is  a  viable  frame- 
work for  the  industry  to  implement 
on  a  test  basis. 

11/41% 

11/41% 

4/15% 

1/4% 

0/0% 

0/0% 

b.  The  workshop  format  offers  a  sound 
approach  for  producing  a  generic  HACCP 
model  for  other  meat  and  poultry  products. 

12/44% 

11/41% 

3/11% 

1/4% 

1/4% 

0/0% 

c.  The  presence  of  Observers  was  beneficial 
to  the  workshop. 

2/8% 

4/17% 

9/38% 

3/13% 

1/4% 

5/21% 

d.  The  workshop  resulted  in  a  useful  generic 

10/37% 

14/52% 

2/7% 

1/4% 

0/0% 

0/0% 

HACCP  model  developed  by  industry 
participants. 
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WORKSHOP  MATERIALS  (Observers:  Please  skip  Question  8  and  continue  with  Question  9.) 


Overall,  how  useful  were  the  written  materials  for: 

a.  Assessing  finished  product  hazards? 

b.  Assessing  ingredient  hazards? 

c.  Determining  placement  of  CCPs  on  flow 
diagram? 

d.  Determining  critical  limits  for  each  CCP? 

e.  Developing  assigned  CCP  monitoring  plan? 

f .  Developing  corrective  action  approaches? 

g.  Developing  record-keeping  systems? 

h.  Developing  verification  procedures? 

i.  Developing  a  generic  HACCP  model? 

WORKSHOP  EXPECTATIONS 


Very 
Useful 

Useful 

Somewhat 

II-  -X.  / 

Mot 
Useful 

Mot  At  All 
Useful 

8/29% 

16/57% 

4/14% 

0/0% 

0/0% 

6/21% 

13/46% 

8/29% 

0/0% 

1/4% 

9/32% 

14/50% 

3/11% 

1/4% 

1/4% 

a  it  Ate 
4/l47e 

i  n ci  a 

17/Ol7c 

0/Zl7c 

0/0% 

1/4% 

4/15% 

18/67% 

5/19% 

0/0% 

0/0% 

4/15% 

17/63% 

5/19% 

J/  1/  /C 

0/0% 

1/4% 

6/22% 

14/52% 

6/22% 

0/0% 

1/4% 

6/22% 

15/56% 

5/19% 

0/0% 

1/4% 

8/31% 

14/54% 

4/15% 

0/0% 

0/0% 

Yes: 

26/93% 

No: 

2/7% 

Did  the  workshop  accomplish  what  you  expected? 
Please  explain. 

(18  comments) 

(5)   Yes.  I  now  have  a  much  better  understanding  of  what  HACCP  is,  how  to  develop  a  specific  plan, 
and  how  to  implement  it  in  the  field.  I  now  know  the  proper  steps  to  develop  a  HACCP  program 
and  material  for  training  at  my  facility.  I  wanted  to  know  where  FSIS  was  in  the  HACCP  study  and 
if  I  could  implement  the  plan  in  my  plant  as  FSIS  may  require.  Basically  I  came  here  to  get  an 
understanding  of  HACCP  and  how  it  might  be  implemented.  The  generic  model  and  the  steps  to 
achieve  it  accomplished  these  goals.  It  showed  me  a  lot  of  work  and  research  have  to  go  into  it. 

(4)   Yes.  We  accomplished  our  goal  of  developing  a  generic  HACCP  model  for  ground  beef  patties. 

(2)  Yes.  Very  helpful.  Concept  outstanding.  I  strongly  support  this  process  that  is  based  on  (1)  TQM 
principles,  (2)  team-building,  idea  generation,  consensus-building;  and  (3)  beneficial  cooperation 
between  government  and  industry.  Thank  you. 

(2)   Yes.  I  think  the  workshop  opened  up  more  of  what  HACCP  is  about  and  how  it  affects  individual 
companies.  I  now  have  a  more  positive  attitude  toward  the  program.  It  should  give  each  plant  a 
strong  skeleton  to  build  on  and  to  develop  beyond  minimum  wholesome  requirements. 

(2)   Na  Very  general.  I  could  have  completed  this  at  the  office.  The  HACCP  model  developed  was 
extremely  vague.  Industry's  apprehensions  about  the  HACCP  system  are  clearly  reflected  in  the 
model  developed.  I  realize  that  many  issues  addressed  will  be  plant-specific  and  will  vary,  however, 
what  was  produced  could  not  be  taken  home  and  implemented  as  is.  I  believe  there  is  still  too  much 
opportunity  for  companies  to  avoid  the  underlying  purpose  of  HACCP:  to  produce  wholesome,  safe 
products. 
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(1) 

Yes.  In  the  first  USDA  HACCP  meeting  I  attended  a  couple  of  years  ago,  I  felt  that  USDA  did  not 
have  a  good  grasp  of  HACCP  concepts,  but  now  they  seem  well-versed  on  it  and  knowledgeable. 

(1) 

Yes.  The  workshop  accomplished  more  than  what  I  had  expected! 

(1) 

Yes.  It  gave  me  an  understanding  of  the  direction  to  use  in  determining  CCPs.  I  do  not  produce  the 
products  used  in  this  workshop,  however,  1  will  be  able  to  adapt  the  model  to  fit  my  operations. 

10.   Did  the  individual  sessions  accomplish  what  you      Yes  for                  Yes  for                No  for 
expected?  Please  explain.                                  All:     21/75%          Some:  7/25%         AH:  0/0% 

(15  comments) 

(5) 

Yes  for  All  The  informal  discussions  gave  us  a  chance  to  explain  our  viewpoints,  answer  questions  and 
focus  more  closely  on  certain  things  that  were  glossed  over  in  the  large  group.   Good  use  of  different 
ways  of  developing  a  HACCP  plan,  small  group,  large  group,  facilitators,  group  leaders,  recorders, 
etc.  The  brainstorming  aspect  worked  very  efficiently. 

(3) 

Yes  for  All  Things  were  well-organized;  this  was  a  big  part  of  getting  things  accomplished.  The 
workshop  was  kept  on  schedule.  The  structure  prevented  domination  by  a  few  people. 

(2) 

Yes  for  All  The  workshop  took  into  account  concerns  for  different  company  sizes  for  HACCP  pro  cram 
development.  It  provided  an  opportunity  for  individuals  in  industry  to  interact  and  to  work  to  obtain 
a  common  goal. 

(2) 

Yes  for  All  At  times  it  seemed  like  the  process  was  overwhelming,  but  the  effort  paid  off  in  the  final 
product  We  developed  principles  that  could  easily  be  adapted  into  industry. 

(1) 

Yes  for  All  I  learned  a  lot  about  what  HACCP  is  and  was  able  to  determine  that  I  could  not  implement 
a  HACCP  plan  in  my  plant  for  financial  and  practical  reasons  and  that  many  other  plants  have  the 
same  problem. 

(1) 

Yes  for  All.  I  gained  a  basic  understanding  of  HACCP  and  program  development. 

(1) 

Yes  for  Some.  The  primary  accomplishment  of  this  workshop  was  one  of  showing  industry  workers, 
first-hand,  what  is  involved  in  the  development  and  implementation  of  such  a  program.  That  was 
very  valuable.  Also,  for  the  most  part,  it  is  positively  perceived  that  government  has  given  industry 
the  opportunity  to  work  through  this  process  together. 

COMMENTS 

11.  Methods  other  than  a  workshop  format  could  have  been  employed  to  develop  a  generic  HACCP  model. 
Examples  of  options  include  an  FSIS  request  for  (a)  comments  on  an  FSIS-generated  model  published  in 
the  Federal  Register;  (b)  development  of  a  model  by  selected  academicians;  or  (c)  submission  to  FSIS  of 
independently-generated  models.  Please  indicate  your  preference  for  one  of  the  above-listed  options, 
or  describe  any  others  not  mentioned  that  you  would  like  to  be  considered  by  FSIS  in  the  future. 

(22  comments) 

Concerning  use  of  a  workshop  format: 

(8) 

Workshop  format  was  excellent.  Workshop  format  was  ideal  to  develop  a  HACCP  model.  The 
workshops  are  the  best  way  to  do  these  programs. 
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(3)    Use  the  workshop  format.  Although  arduous  to  "quick  results"-  oriented  industry,  the  process  is 
important  so  that  everyone  has  buv-in  to  the  end  product  of  the  entire  group.  The  workshop  format 
informs  better  and  lets  individuals  work  together  to  find  a  happy  medium  that  works  for  all  sizes  of 
companies.  The  present  format  has  industry  and  USD  A  working  together.  It  is  not  having  someone 
else  develop  a  program  and  then  tell  everyone  to  make  it  work.  This  way  all  HACCP  programs  follow 
the  same  format. 

(2)   This  was  by  far  the  best  method.  It  had  everyone  present  so  everything  was  aired.  The  other  methods 
would  have  resulted  in  "back  and  forth"  discussion  "forever"  between  industry  and  government. 
Workshops  as  this  one  save  a  good  deal  of  "flack"  from  industry. 

(2)    None  of  the  options  above.  Models  are  best  built  through  idea  generation.  I  suggest  that  a  follow-up 
workshop  be  held  after  the  pilot  program  to  review  that  work  and  to  discuss  in  an  open  forum  wrhat 
modifications  to  make  to  a  preliminary  model. 

(2)   Prefer  the  workshop  method.  Second  choice  would  be  Option  (C):  submission  to  FSIS  of 
independently-generated  models. 

Concerning  other  options: 

(2)   Option  (A):  comments  on  an  FSIS- genera  ted  model  published  in  the  Federal  Rsgister.  Of  these  options, 
I  believe  comments  on  an  FSIS  model  may  work.  Comments  should  come  from  as  many  different 
groups  as  possible,  not  industry  alone. 

(2)   Option  (C):  submission  of  independently-developed  models. 

(1)    Option  (B):  development  of  a  model  by  selected  academicians. 


12.  What  was  most  beneficial  to  you  about  the  workshop? 
(23  comments) 

(6)   Getting  involved!  Cooperation  between  processors  and  FSIS —  it  was  great!  Interaction,  comments 
and  input  from  representatives  of  all  sectors.  Having  both  industry  and  governmental  input  to 
develop  a  HACCP  model.  Meeting  other  members  of  industry  and  meeting  persons  responsible  for 
decision-making. 

(5)   Insight  into  the  purpose  of  HACCP.  Learning  what  HACCP  is  and  what  it  may  lead  to.  Understanding 
HACCP  and  its  development  and  implementation.  The  steps  involved  in  the  development  of  the 
HACCP  program  and  the  materials  made  available  for  further  training  when  I  return  to  my  facility. 

(4)   Group  participation.  Participation  in  small  group  sessions  with  reports  to  large  groups  was  most 
beneficial. 

(3)   Sharing  ideas  and  problems  that  face  all  of  us  to  produce  safe  product.  The  interaction  between  the 
Participants  and  also  between  the  Participants  and  the  FSIS  staff.  The  more  different  perspectives  that 
can  be  shared  and  evaluated,  the  more  useful  the  session.  The  opportunity  to  provide  input  by  stating 
my  own  views,  concerns  and  desires. 

(3)   Well-organized,  good,  necessary  information.  Helps  give  an  idea  of  what  is  happening  in  other  areas 
of  the  country  regarding  regulatory  issues. 


-5- 


(1 )    Observing  politics  in  action! 

(1)  I  was  able  to  determine  that  it  is  impractical  and  unnecessary  in  a  small  plant. 

13.  What  was  least  beneficial  to  you  about  the  workshop? 
(14  comments) 

(3)   Nothing.  Everything  was  necessary. 

(3)   Big  discussion  over  small  words.  Too  much  discussion  of  "words"  in  development  of  the  model. 
This  should  be  handled  by  the  Steering  Committee. 

(2)  The  first  morning.  Remarks  at  the  opening  of  the  workshop. 
(1)   Too  slow. 

(1 )  I  think  we  could  have  eliminated  the  first  part  of  assessing  CCPs  in  the  flow  diagram.  Going  back  to 
eliminate  whether  there  actually  were  risks  involved  and  deleting  some  seemed  like  an  unnecessary 
step. 

(1 )   Too  much  discussion  of  individual  plant  processes. 

(1)   Most  FSIS  people  stayed  together  and  we  didn't  get  time  to  talk  one-on-one,  or  in  a  less  formal  session. 

(1 )   Observers  and  too  many  government  personnel  with  different  opinions  about  how  programs  should 
be  developed  and  implemented. 

(1 )  Voting  on  the  same  items  over  and  over. 

14.  Considering  the  agenda  for  the  first  morning  of  the  workshop,  should  any  of  the  introductory  activities  or 
presentations  be  expanded,  shortened,  or  modified  as  to  content,  to  better  meet  your  needs?  Please  explain. 

(19  comments) 

(9)  The  agenda  seemed  acceptable  to  me.  What  was  presented  was  adequate.  The  first  morning  set  the 
stage  for  the  remainder  of  the  workshop.  I  would  not  expand  or  add  anything.  Just  fine.  I  needed 
time  to  understand  the  objectives  and  methodology.  Once  those  were  clear,  it  became  easy  to  be 
productive  during  discussions. 

(6)  The  first  session  was  too  slow.  It  should  be  shortened.  Introduction  was  too  long.  All  of  the 
introduction  should  be  shortened  to  a  brief  overview  of  HACCP  by  one  individual.  History  not 
needed.  Some  of  the  "history  lesson"  could  have  been  excluded.  I  would  say  it  was  marginally 
beneficial —  I  was  already  familiar  with  a  lot  of  the  information. 

(1 )   First  morning's  agenda  was  more  than  adequate! 

(1)   Define  HACCP  and  give  an  overview  of  how  a  program  is  set  up  prior  to  jumping  right  into  setting 
one  up. 

(1)  Emphasize  that  Participants  should  study  materials  prior  to  the  workshop  and  shorten  the  first 
morning's  introductory  activities. 

(1 )   USD  A  people  did  a  good  job  of  making  you  welcome,  but  more  time  should  have  been  allowed  for 
everyone  to  get  to  know  each  other. 
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15.   Please  use  the  rest  of  this  page  to  make  any  additional  comments  that  may  help  us  understand  your 
thoughts  about  the  worksnop  and  improve  future  activities. 

(28  comments) 

Workshop-related: 

(6)   Good  Show!!  Things  were  organized  and  planned  well  and  moved  along  well  for  such  a  diversified 
group.  It  was  good  to  form  policies  with  industry  and  USDA  personnel  input.  You  get  a  well- 
balanced  decision.  Very  good  idea  to  let  industry  write  guidelines  that  they  would  be  comfortable 
with.  We  must  work  together.  The  cross-section  of  size  of  companies  represented  was  good  and 
important  so  that  the  model  is  not  skewed  to  large  vs.  small  (or  vice  versus)  companies.  I  appreciate 
the  opportunity  to  be  involved.  Keep  up  the  open  and  friendly  relationship  between  industry  and 
USDA. 

(3)   Not  good  to  have  previous  models  available.  Do  not  give  us  the  comments  of  other  workshops.  Let 
us  get  our  own  ideas.  Previous  models  should  not  be  passed  out  until  the  end  of  the  session.  I  believe 
you  would  get  more  additional  input,  rather  than  copies  of  what  the  last  workshop  thought  up. 

(1)   It  really  helped  a  lot  that  we  had  access  to  the  output  of  other  workshops.  We  could  creatively  add  to 
their  work  rather  than  waste  time  re-creating  the  same  steps. 

(1 )   The  workshop  did  a  very  good  job  of  showing  the  steps  involved  in  developing  a  HACCP  program. 
It  also  provided  material  for  training  of  employees  when  we  start  our  implementation. 

(1 )   The  second  day's  listing  of  all  potential  CCPs  then  going  back  and  reviewing  seemed  redundant. 

(1)   It  would  be  useful  to  have  a  single,  short  document  to  refer  to  during  discussions  rather  than  having 
to  search  for  reminders  of  points  made  earlier. 

(1 )  Provide  more  information  on  volunteer  plants  for  testing  HACCP  program. 

(1)  The  workshop  could  be  handled  more  quickly  if  a  test  model  were  developed  prior  to  the  workshop 
and  discussed. 

( 1 )   Stress  the  importance  of  generic. 

(1)  The  program  was  dominated  by  a  small  group  of  people —  especially  by  those  who  are  not  in  the 
industry  of  producing  ground  beef. 

HACCP-related: 


(1 )  This  program  is  just  a  fancy  name  for  programs  that  our  plants  are  presently  doing. 

(1)  I  started  out  employing  three  people  and  was  a  laborer  myself  as  well  as  owner/manager.  As  the 
plant  grew  larger  record-keeping  became  proportionally  important.  (The  larger  the  plant  got  the 
more  record-keeping.)  But  with  100  current  employees  we  do  not  need  near  the  paperwork  that 
HACCP  would  generate.  I  can  see  where  it  would  apply  in  a  very  large  plant  (more  than  500 
employees),  but  for  anything  smaller  it  is  impractical.  The  largest  difference  between  what  is 
proposed  and  our  current  procedure  is  we  record  only  the  defects.  If  everything  is  OK  we  do  not 
keep  any  record.  This  greatly  reduces  expenses. 
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Other  ideas: 


(2)    Why  so  many  government  people?  This  was  a  common  question  by  many  of  the  Participants. 

Excessive  number  of  FSIS  employees  here  who  had  no  role  in  the  workshop.  Why  were  they  here? 
Money  was  wasted! 

(2)    Arrange  the  material  in  the  workbook  in  a  more  orderly  fashion.  Need  page  numbers  on  overhead 
sheets  and  on  pages  in  the  workbook. 

(1)   Save  space  and  paper  by  re-setting  the  overhead  charts  into  bullet  points  typed  on  fewer  pages.  I  don't 
need  the  type  size  to  be  1  IT  when  it's  right  in  front  of  me. 

(1 )   This  may  sound  a  little  picky,  but  the  name  tags  were  totally  useless  because  the  printing  was  hard  to 
read  against  the  colored  background  and  it  was  hard  to  tell  who  you  were  talking  to  during  breaks. 
Bold  lettering  against  a  white  background  makes  more  sense. 

(1 )   Many  Participants  did  not  receive  the  materials  mailed  concerning  the  workshop  prior  to  the  workshop. 
That  started  things  off  in  a  negative  light. 

(1)   My  frustration  was  that  there  wasn't  more  emphasis  on  education.  This  large  a  group  would  have  been 
a  good  forum  for  such  educational  interaction,  for  example:  specifics  on  metal  detection  procedures. 

(1 )   Great  facility;  terrible  sound. 
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SECTION  II—  OBSERVERS 


SUMMARY  OF  OBSERVERS'  RESPONSES 
EVALUATION  QUESTIONNAIRE 


orkshop  lor  1  i  ALL. J'  Model  1  development  tor 
Fresh  Ground  Beef,  December  3-5, 1991 


SUMMARY  OF  OBSERVERS'  RESPONSES  * 

RESPONDENT  CHARACTERISTICS 

1 .     Which  grouping  below  best  characterizes  your  current  affiliation? 

Meat  &  Poultry  Industry:  1/17% 

Other  Federal  Government:  2/33% 

Other:  2/33% 

Unspecified:  1/17% 

2.     If  you  marked  "Meat  &  Poultry  Industry"  in  Question  1,  please  indicate  all  groupings  below  which  best 
characterize  your  industry  type  and  the  total  production  volume  of  your  establishment(s).  [Low  Volume 
Establishment  =  1  -  59,999  lbs.  produced/Quarter;  Medium  Volume'  Establishment  =  60,000  - 1,000,000  lbs. 
produced  /Quarter;  High  Volume  Establishment  =  more  than  1,000,001  lbs.  produced  /Quarter] 

Meat  &  Poultry  Industry                Low  V  olume  Establishment(s): 

Medium  Volume  Establishment(s): 
High  Volume  Establishment(s): 

0/0% 
0/0% 
1/100% 

3.     Days  attended  (mark  all  that  apply):           1  Day:  0/0%              2  Days:  0/0% 

3  Days: 

6/100% 

4.     Please  check  one:                                    Observer:    6             Participant:  0 

HACCP  MODEL  DEVELOPMENT 

Very                          Somewhat  Not 
Successful  Successful  Successful  Success 

5.     Overall,  how  successful  was  the  workshop  in: 

Very 
ful  Unsuccessful 

Don't 
Know 

a.  Identifying  Critical  Control  Points  (CCPs)?  3/50%      1/17%      1/17%  1/17% 

0/0% 

0/0% 

b.  Determining  critical  limits  for  each  CCP?     1/17%      2/33%      3/50%  0/0% 

0/0% 

0/0% 

c.  Developing  monitoring  plan  activity  for      1/17%      2/33%      3/50%  0/0% 
identified  CCPs? 

0/0% 

0/0% 

d.  Developing  corrective  action  approaches?   1/17%      3/50%      2/33%  0/0% 

0/0% 

0/0% 

e.  Developing  record-keeping  systems?         1/17%      2/33%      3/50%  0/0% 

0/0% 

0/0% 

f.  Developing  verification  procedures?          1/17%      2/33%      3/50%  0/0% 

0/0% 

0/0% 

g.  Developing  a  generic  HACCP  model?        1/17%      4/67%      1/17%  0/0% 

0/0% 

0/0% 

NOTE:  Results  are  presented  as  number  of  responses!  %  of  responses  for  each  category. 

*  Due  to  the  very  small  number  of  observers,  percentages  may  be  misleading. 
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DESIGN  AND  OPERATION  OF  WORKSHOP 

More  than 
Adequate 

Adequate 

Marginally 
Adequate 

Inadequate 

Very 

Inadequate 

Don't 
Know 

6. 

Please  rate  the  workshop  in  terms  of: 

a. 

Development  of  a  generic  product /process 
acscnption 

1/17% 

5/83% 

0/0% 

0/0% 

0/0% 

0/0% 

h 

Dr»vplr»T"»rnf»r»  f  r\f  nrnroQQ  flow  Hiactvim 

1/17% 

5/83% 

0/0% 

0/0% 

0/0% 

0/0% 

c. 

Organization  of  the  three  days 

4/67% 

2/33% 

0/0% 

0/0% 

0/0% 

0/0% 

d. 

Use  of  small  "break-out"  groups 

4/67% 

1/17% 

0/0% 

1/17% 

0/0% 

0/0% 

e. 

Reporting  of  small  "break-out"  group 
activities  to  entire  group 

4/67% 

2/33% 

0/0% 

0/0% 

0/0% 

0/0% 

f. 

Opportunity  for  input  from  Observers 

2/33% 

1/17% 

0/0% 

1/17% 

2/33% 

0/0% 

g- 

Opportunity  for  input  from  Participants 

2/33% 

3/50% 

0/0% 

1/17% 

0/0% 

0/0% 

n. 

erreenveness  or  racintators  in  neipmg 
achieve  workshop  objectives 

5/83% 

1/17% 

0/0% 

0/0% 

0/0% 

0/0% 

i. 

Effectiveness  of  subject  matter  experts  in 
helping  achieve  workshop  objectives 

3/50% 

2/33% 

0/0% 

0/0% 

0/0% 

1/17% 

j- 

Effectiveness  of  group  leaders  in  helping 
achieve  workshop  objectives 

4/67% 

2/33% 

0/0% 

0/0% 

0/0% 

0/0% 

k. 

Effectiveness  of  group  reporters  in  helping 
achieve  workshop  objectives 

3/50% 

2/33% 

1/17% 

0/0% 

0/0% 

0/0% 

1. 

Completion  of  workshop  objectives. 

3/50% 

3/50% 

0/0% 

0/0% 

0/0% 

0/0% 

7. 

Please  indicate  your  degree  of  agreement 
with  each  of  the  following  statements. 

Strongly 
Agree 

Agree 

Neutral 

Disagree 

Strongly 
Disagree 

Don't 
Know 

a. 

The  HACCP  model  developed  by  the 
workshop  participants  is  a  viable  frame- 
work for  the  industry  to  implement 
on  a  test  basis. 

2/33% 

2/33%) 

1/17% 

1/17% 

0/0% 

0/0% 

b. 

The  workshop  format  offers  a  sound 
approach  for  producing  a  generic  HACCP 
model  for  other  meat  and  poultry  products. 

2/33% 

4/67% 

0/0% 

0/0% 

0/0% 

0/0% 

c. 

The  presence  of  Observers  was  beneficial 
to  the  workshop. 

1/17% 

3/50% 

1/17% 

1/17% 

0/0% 

0/0% 

d. 

The  workshop  resulted  in  a  useful  generic 
HACCP  model  developed  by  industry 
participants. 

2/33% 

2/33% 

2/33% 

0/0% 

0/0% 

0/0% 

-12- 


WORKSHOP  MATERIALS  (Observers:  Please  skip  Question  8  and  continue  with  Question  9.) 

Very  Somewhat     Not  Not  At  All 

Useful  Useful         Useful  Useful  Useful 

8.     Overall,  how  useful  were  the  written  materials  for: 

a.  Assessing  finished  product  hazards? 

b.  Assessing  ingredient  hazards? 

c.  Determining  placement  of  CCPs  on  flow 
diagram? 

d.  Determining  critical  limits  for  each  CCP?  OBSERVERS  WERE  INSTRUCTED  TO 

e.  Developing  assigned  CCP  monitoring  plan?  SKIP  THIS  QUESTION. 

f .  Developing  corrective  action  approaches? 

g.  Developing  record-keeping  systems? 

h.  Developing  verification  procedures? 

i.  Developing  a  generic  HACCP  model? 


WORKSHOP  EXPECTATIONS 

9.     Did  the  workshop  accomplish  what  you  expected?  Yes:  3/50%  No:  3/50% 

Please  explain. 

(8  comments) 

(3)    Yes.  I  learned  much  about  how  HACCP  operates.  The  HACCP  Committee  did  an  excellent  job  and 
worked  hard  to  help  the  Participants  come  up  with  a  model.  The  generic  model  was  generated 
through  input  from  industry  representatives. 

(2)   Ncl  I  anticipated  some  solid  guidelines  to  be  produced  by  industry  as  to  what  was  "adequate"  for  their 
own  self-regulation.  Instead  I  found  a  sense  of  "fear7'  that  the  small  plant  operators  couldn't  comply 
with  any  type  of  a  minimum  guideline,  whether  it  be  a  temperature  or  time  length  or  amount  of 
foreign  material.  Some  Participants  are  looking  at  FSIS  as  an  enemy  rather  than  as  an  ally. 

(2)   Na  I  felt  restrained  from  offering  input  which  might  have  contributed  to  the  process.  There  should 
have  been  time  allotted  for  the  Observers  to  comment —  perhaps  a  few  minutes  at  the  end  of  each 
workshop  or  session,  even  though  we  could  not  vote,  etc.  I  believe  outside  input  would  still  be 
beneficial. 

(1)   Ncl  I  really  felt  the  Participants  would  be  more  critical  of  themselves  and  come  up  with  more  CCPs. 
I  felt  each  Participant  was  looking  out  for  their  individual  plants,  and  addressing  their  own  unique 
problems. 
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10.    Did  the  individual  sessions  accomplish  what  you      Yes  for  Yes  for  No  for 

expected?  Please  explain.  All:     1/20%  Some:  4/80%         All:  0/0% 

(5  comments) 

(1)    Yes  for  All.  Individual  groups  accomplished  assigned  tasks  effectively  and  efficiently. 

(1 )    Yes  for  Some.  Some  good  brainstorming,  a  great  sense  of  camaraderie  between  the  teams,  good 

negotiations  for  a  consensus.  However,  no  one  felt  comfortable  enough  to  identify  a  specific  guideline 
and  be  able  to  justify  it  to  the  entire  workshop. 

(1)    Yes  for  Some.  I  felt  there  was  some  confusion  by  some  Participants  as  to  what  the  goals  were.  Other 
Participants  understood  what  the  purpose  of  developing  a  HACCP  model  was  all  about. 

(1)    Yes  for  Some.  I  was  not  able  to  observe  enough  of  the  small  group  sessions. 

(1)    Yes  for  Some.  There  were  times  when  I  felt  like  the  speakers  had  almost  put  the  Participants  to  sleep. 
Presentation  dynamics  are  very  important. 


COMMENTS 

1 1 .   Methods  other  than  a  workshop  format  could  have  been  employed  to  develop  a  generic  HACCP  model. 
Examples  of  options  include  an  FSIS  request  for  (a)  comments  on  an  FSIS-generated  model  published  in 
the  Federal  Register;  (b)  development  of  a  model  by  selected  academicians;  or  (c)  submission  to  FSIS  of 
independently-generated  models.  Please  indicate  your  preference  for  one  of  the  above-listed  options, 
or  describe  any  others  not  mentioned  that  you  would  like  to  be  considered  by  FSIS  in  the  future. 

(4  comments) 

Concerning  use  of  a  workshop  format: 

(2)   Use  the  workshop  format.  I  was  very  satisfied  with  the  workshop  format  as  it  provided  a  great 
amount  of  feedback  from  all  Participants  and  allowed  lots  of  open  communication.  A  government- 
generated  model  would  not  have  had  as  much  discussion  as  to  what  could  be  accomplished  by  the 
small  operator.  It  seemed  they  were  the  greatest  concern. 

Concerning  other  options: 

(1 )   If  the  workshop  method  is  not  used,  I  would  prefer  development  by  a  small  independent  group 
selected  from  academia,  industry  and  government. 

(1)   Option  (B).  A  generic  model  should  have  been  developed  and  presented  to  the  workshop  Participants 
for  their  modification,  as  needed. 
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12.    What  was  most  beneficial  to  you  about  the  workshop? 
(6  comments) 

(2)    The  actual  model  which  was  generated.  Explanation  of  the  HACCP  process  and  model  development 
in  the  large  group  sessions. 

(2)    I  liked  the  organization  and  the  format  of  the  workshop  and  appreciated  the  efforts  made  to  keep  on 
schedule.  The  HACCP  Committee  was  very  organized  and  knew  what  direction  they  were  going. 
I  know  it  is  hard  to  develop  a  program  or  model  with  so  many  different  ideas. 

(2)   To  observe  the  attitudes  and  actions  of  industry  "people"  in  their  own  realm —  without  motivation 
provided  by  inspections.  I  really  got  a  feel  for  how  industry  "bands  together"  to  look  after  each  other. 


13.   What  was  least  beneficial  to  you  about  the  workshop? 
(6  comments) 

(2)   I  was  disappointed  to  see  that  when  industry  had  the  opportunity  to  set  some  "general"  guidelines 
they  were  very  resistant  to  generate  any  numbers  or  generic  procedures.  "Plant-specific"  did  not 
identify  any  common  procedure  or  method  any  plant  could  put  in  place.  The  Participants  did  not 
really  understand  the  real  purpose  of  the  workshop.  Possibly  they  were  protecting  their  turf.  I  also 
believe  the  Participants  did  not  feel  that  HACCP  is  important.  They  were  reluctant  to  identify  CCPs. 

(2)   The  lack  of  participation  for  Observers.  I  felt  they  should  have  been  informed  that  they  could  comment 
and  when  those  comments  could  be  voiced.  Observers  should  have  some  input  or  be  able  to  make 
some  comments. 

(1)   As  an  Observer,  the  breakout  into  small  group  sessions  did  not  allow  me  to  follow  what  was  really 
happening. 

(1)   The  history /background  on  ground  beef  was  least  beneficial. 


14.  Considering  the  agenda  for  the  first  morning  of  the  workshop,  should  any  of  the  introductory  activities  or 
presentations  be  expanded,  shortened,  or  modified  as  to  content,  to  better  meet  your  needs?  Please  explain. 

(5  comments) 

(2)   The  introductory  activities  as  conducted  (not  as  scheduled)  appeared  to  be  appropriate.  The  agenda 
seemed  appropriate  for  the  intended  purpose. 

(1)  The  HACCP  Committee  should  be  more  direct  and  control  the  meeting  more.  The  concept  of  "generic 
model"  needs  to  be  made  more  clear.  The  Participants  were  trying  to  put  their  own  needs  or  concerns 
into  the  model  instead  of  coming  up  with  a  generic  model. 

(1 )   Shorten  the  instructions  given  during  the  introduction  and  spend  more  time  on  specific  plant-type 
operations  to  identify  a  true  critical  control  point. 

(1)   Shorten  the  history  /background  on  ground  beef;  add  history /background  on  HACCP. 
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15.    Please  use  the  rest  of  this  page  to  make  any  additional  comments  that  may  help  us  understand  your 
thoughts  about  the  workshop  and  improve  future  activities. 

(5  comments) 

(1)  There  arc  obviously  more  efficient  ways  to  develop  the  models,  however,  the  educational  benefit  of  the 
workshop  approach  appears  to  be  appropriate  for  this  activity.  It  will  certainly  enhance  the  chances  of 
acceptance  of  HACCP  by  industry  and  inspectors  if  it  is  implemented. 

(1)    I  was  surprised  to  see  how  many  potential  CCPs  were  not  identified  by  industry.  They  felt  any  type 
of  water  was  acceptable  without  testing  and  assumed  many  incoming  products  were  OK —  sight 
unseen.  They  all  felt  the  plant  workers  never  varied  from  the  best  of  Good  Management  Practices  and 
so  therefore  no  checks  and  balances  were  needed  as  processing  progressed.  It  seemed  as  though  no 
one  wanted  any  unnecessary  burdens  placed  upon  any  company — always  citing  the  small  plants. 

(1)   This  workshop  reemphasized  more  than  ever  to  me  why  meat  and  poultry  inspection  is  needed. 
I  enjoyed  the  proceedings,  not  the  Participants'  indecision.  FSIS  or  the  HACCP  Committee  should 
have  developed  the  HACCP  model  and  left  the  Participants  out  of  developing  the  model.  Maybe 
some  consumer  groups  also  could  have  helped  formulate  the  HACCP  model.  The  Participants  were 
unwilling  to  face  up  to  this  responsibility.  Also,  there  should  be  one  or  two  HACCP  Committee 
members  who  work  for  State  programs.  State  participation  on  the  Committee  was  nil.  Remember 
most  states  pay  for  50%  of  the  programs  in  their  states. 

(1)   The  use  of  State  Talmadge/ Aiken,  USDA  inspectors  should  be  considered  for  future  HACCP  or 
other  such  Federal  programs  since  USDA  is  helping  with  the  funding  and  all  Talmadge/ Aiken 
inspectors  should  be  USDA-certified  as  being  "equal  to"  Federal  inspectors,  thus  qualifying  them 
for  participation  in  all  areas. 

(1)   FSIS  staff  was  most  friendly  and  helpful;  facilities  were  excellent.  Organization  of  workshop  also 
was  excellent  and  the  notebook  of  materials  and  the  handouts  were  very  helpful.  Thank  you  for  the 
opportunity  to  observe! 
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Summary  of  the 
Evaluation  Results  of  the 
Workshop  for  HACCP 
Model  Development 


Minneapolis,  Minnesota,  March  31  -  April  2,  1992 


INTRODUCTION 


This  report  presents  the  evaluation  results  from  the  Food  Safety  and  Inspection  Service  (FSIS) 
of  the  United  States  Department  of  Agriculture  (USD A)  workshop  to  develop  a  generic 
model  for  a  Hazard  Analysis  and  Critical  Control  Point  (HACCP)  plan  for  swine  slaughter. 
Representatives  from  industry,  trade  associations  and  government  met  in  Minneapolis, 
Minnesota,  March  31  -  April  2, 1992,  for  this  workshop  which  focused  on  market  hogs.  The 
workshop  was  open  to  the  public  for  observation,  on  a  reserved  space  basis. 

Participants  and  Observers  were  asked  to  complete  a  questionnaire  which  asked  them 
questions  related  to  the  design  and  operation  of  the  workshop,  the  workshop  materials,  the 
process  employed  in  developing  a  generic  HACCP  model,  and  their  views  concerning  the 
workshop  approach  and  any  suggested  improvements.  Of  the  30  Participants  and  6  Observers 
in  attendance,  30  Participants  (100%)  and  2  Observers  (33%)  responded  to  the  questionnaire. 

This  evaluation  activity  is  one  in  a  series  of  on-going  evaluation  tasks  that  are  a  part  of  the 
HACCP  Implementation  Study.  The  results  of  this  evaluation  should  serve  to  help  plan 
subsequent  workshops  and  to  provide  background  information  to  aid  in  future  planning 
decisions  related  to  HACCP. 


REPORT  PLAN 


This  report  is  organized  into  two  sections.  Section  I  presents  the  results  and  summary  of 
Participants'  responses  and  Section  II  presents  those  for  the  Observers.  Results  of  the 
multiple-choice  questions  are  presented  in  a  format  based  on  that  of  the  original  questionnaire. 
The  responses  are  reported  by  number  and  percent  of  responses  for  each  category.  Percentages 
reported  are  based  on  the  number  of  responses  to  each  question  and  do  not  always  total 
100  percent  due  to  arithmetic  rounding. 

Results  of  the  open-ended  questions  are  also  summarized  in  each  Section.  A  content  analysis 
was  performed  for  all  hand-written  comments.  Respondents'  remarks  are  paraphrased 
and  grouped  by  question.  Some  respondents  made  multiple  comments  while  others  made 
none,  thus  the  number  of  comments  per  question  does  not  equal  the  number  of  respondents. 
The  number  of  respondents  who  concurred  with  each  comment  is  listed  parenthetically. 


SECTION  I—  PARTICIPANTS 


SUMMARY  OF  PARTICIPANTS'  RESPONSES 
EVALUATION  QUESTIONNAIRE 


Workshop  for  HACCP  Model  Development  for 
Swine  Slaughter,  March  31  -  April  2, 1992 


SUMMARY  OF  PARTICIPANTS'  RESPONSES 
RESPONDENT  CHARACTERISTICS 

1.    Which  grouping  below  best  characterizes  your  current  affiliation? 

Meat  &  Poultry  Industry:  28/93% 
Educational  Institution:  1/3% 
Other  Government:  1/3% 


If  you  marked  "Meat  &  Poultry  Industry"  in  Question  1,  please  indicate  all  groupings  below  which  best 
characterize  your  industry  type  and  the  total  production  volume  of  your  establishment(s).  [Low  Volume 
Establishment  =  1  -  59,999  lbs.  produced /Quarter;  Medium  Volume  Establishment  =  60,000  - 1,000,000  lbs. 
produced /Quarter;  High  Volume  Establishment  =  more  than  1,000,001  lbs.  produced /Quarter] 


Meat  Industry 


Low  Volume  Establishments): 
Medium  Volume  Establishments): 
High  Volume  Establishments): 
Volume  Unspecified: 


0/0% 
2/7% 
25/89% 
1/4% 


3.  Days  attended  (mark  all  that  apply): 

4.  Please  check  one: 

HACCP  MODEL  DEVELOPMENT 


IDay:  3/10% 
Observer:  0 


2  Days:  0/0% 
Participant:  30 


3  Days:  27/90% 


5.    Overall,  how  successful  was  the  workshop  in: 


c  Developing  monitoring  plan  activity  for 
identified  CCPs? 


e.  Developing  record-keeping  systems? 

f .  Developing  verification  procedures? 

g.  Developing  a  generic  HACCP  model? 


Very 

Successful 

Successful 

Somewhat 
Successful 

Not 

Successful 

Very  Don't 
Unsuccessful  Know 

3/10% 

19/63% 

8/27% 

0/0% 

0/0% 

0/0% 

3/10% 

20/67% 

7/23% 

0/0% 

0/0% 

0/0% 

3/10% 

19/63% 

8/27% 

0/0% 

0/0% 

0/0% 

4/13% 

20/67% 

6/20% 

0/0% 

0/0% 

0/0% 

1/3% 

23/77% 

5/17% 

1/3% 

0/0% 

0/0% 

13% 

25/83% 

4/13% 

0/0% 

0/0% 

0/0% 

4/13% 

22/73% 

4/13% 

0/0% 

0/0% 

0/0% 

NOTE:  Results  are  presented  as  number  of  responses/%  of  responses  for  each  category. 


-1 


DESIGN  AND  OPERATION  OF  WORKSHOP 


7. 


Please  rate  the  workshop  in  terms  of: 

More  than 
Adequate 

Adequate 

Marginally 
Adequate 

Inadequate 

Very 

Inadequate 

Don't 
Know 

a.  Development  of  a  generic  product /process  8/28% 
description 

21/72% 

0/0% 

0/0% 

0/0% 

0/0% 

b.  Development  of  process  flow  diagram 

11/38% 

18/62% 

0/0% 

0/0% 

0/0% 

0/0% 

c  Organization  of  the  three  days 

11/37% 

15/50% 

4/13% 

0/0% 

0/0% 

0/0% 

d.  Use  of  small  "break-out"  groups 

17/57% 

13/43% 

0/0% 

0/0% 

0/0% 

0/0% 

e.  Reporting  of  small  T>reak-our  group 
activities  to  entire  group 

9/30% 

20/67% 

1/3% 

0/0% 

0/0% 

0/0% 

f.  Opportunity  for  mput  from  Observers 

3/10% 

12/40% 

7/23% 

5/17% 

2/7% 

1/3% 

g.  Opportunity  for  input  from  Participants 

15/52% 

14/48% 

0/0% 

0/0% 

0/0% 

0/0% 

h.  Effectiveness  of  facilitators  in  helping 
achieve  workshop  objectives 

16/53% 

12/40% 

2/7% 

0/0% 

0/0% 

0/0% 

i.  Effectiveness  of  subject  matter  experts  in 
helping  achieve  workshop  objectives 

8/27% 

16/53% 

6/20% 

0/0% 

0/0% 

0/0% 

j.  Effectiveness  of  group  leaders  in  helping 
achieve  workshop  objectives 

14/47% 

16/53% 

0/0% 

0/0% 

0/0% 

0/0% 

k.  Effectiveness  of  group  reporters  in  helping  7/23% 
achieve  workshop  objectives 

23/77% 

0/0% 

0/0% 

0/0% 

0/0% 

1.  Completion  of  workshop  objectives. 

6/20% 

22/73% 

2/7% 

0/0% 

0/0% 

0/0% 

Please  indicate  your  degree  of  agreement 
with  each  of  the  following  statements. 

Strongly 

Agree 

Neutral 

Disagree 

Strongly 
Disagree 

Don't 
Know 

a.  The  HACCP  model  developed  by  the 
workshop  participants  is  a  viable  frame- 
work for  the  industry  to  implement 
on  a  test  basis. 

3/10% 

23/77% 

3/10% 

1/3% 

0/0% 

0/0% 

b.  The  workshop  format  offers  a  sound 

10/33% 

15/50% 

3/10% 

2/7% 

0/0% 

0/0% 

approach  for  producing  a  generic  HACCP 
model  for  other  meat  and  poultry  products. 

c.  The  presence  of  Observers  was  beneficial  2/7% 
to  the  workshop. 

d.  The  workshop  resulted  i  a  useful  generic  5/17% 
HACCP  model  developed  by  industry 
participants. 


4/13%       17/57%     6/20%  0/0% 


19/63%     5/17%       1/3%  0/0% 


1/3% 


0/0% 
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WORKSHOP  MATERIALS  (Observers:  Please  skip  Question  8  and  continue  with  Question  9.) 


Very 
Useful 

Useful 

Somewhat 
Useful 

Not 
Useful 

Not  A 
Usefu 

8. 

Overall,  how  useful  were  the  written  materials  for: 

a.  Assessing  hazards? 

6/21% 

20/69% 

3/10% 

0/0% 

0/0% 

b.  Identifying  preventive  measures? 

3/10% 

17/59% 

9/31% 

0/0% 

0/0% 

c  Deterrnining  placement  of  CCPs  on  flow 
diagram? 

6/21% 

18/64% 

4714% 

0/0% 

0/0% 

d.  Deterrnining  critical  limits  for  each  CCP? 

4/14% 

1  fi  let  a? 
10/0x7* 

5/1/7* 

2/7  7c 

n/nv 

U/U7* 

e.  Developing  assigned  CCP  monitoring  plan? 

A  n  A  Of 

4/14  7c 

10/377* 

7/Z57* 

1/47* 

U/U7* 

f .  Developing  corrective  action  approaches? 

3/11% 

20/71% 

4714% 

1/4% 

0/0% 

g.  Developing  record-keeping  systems? 

2/7% 

19/68% 

6/21% 

1/4% 

0/0% 

h.  Developing  verification  procedures? 

3/11% 

19/68% 

5/18% 

1/4% 

0/0% 

i.  Developing  a  generic  HACCP  model? 

6/21% 

19/68% 

3/11% 

0/0% 

0/0% 

WORKSHOP  EXPECTATIONS 

9. 

Did  the  workshop  accomplish  what  you  expected? 

Yes: 

26/96% 

No: 

1/4% 

Please  explain. 
(18  comments) 


(6)   Yes.  The  workshop  gave  us  a  basic  understanding  of  the  principles  and  concepts  necessary  to  formulate 
HACCP  plans  and  to  set  up  programs  in  our  own  plants.  It  provided  a  good  working  knowledge  of 
HACCP  and  reinforced  application  of  the  HACCP  approach.  It  made  clear  a  viable  method  to  be  used 
in  a  slaughter  situation  to  identify  and  control  hazards. 

(3)   Yes.  The  workshop  accomplished  what  I  expected  and  maybe  more.  I  was  quite  surprised  at  the  good 
and  positive  attitudes  of  the  HACCP  team  and  the  help  they  and  other  FSIS  personnel  provided.  The 
workshop  was  well-led  and  input  was  encouraged  by  all.  Good  communicative  effort  by  all  group 
members. 

(3)   Yes.  A  generic  HACCP  plan  that  is  general  enough  to  fit  into  any  plant  was  developed.  The  workshop 
was  very  beneficial  for  slaughtering  use,  but  the  same  basics  can  be  used  to  develop  all  products. 

(2)   Yes.  The  development  of  the  HACCP  model  was  done  very  systematically.  This  helped  in  clarifying 
the  process.  The  system  of  computer-generated  handouts  was  very  helpful. 

(1)   Yes.  Since  a  HACCP  program  needs  to  be  plant-specific,  we  did  the  best  we  could  in  this  workshop  at  a 
generic  level. 

(1)   Yes.  Tl  s  was  much  much  better  than  the  cooked  sausage  workshop. 

(1)  Yes.  My  expectations  were  to  understand  what  USDA/FSIS's  intent  is  in  regard  to  HACCP  and  to 
witness  the  development  of  a  HACCP  model. 

(1)  Ngi  Although  I  understand  a  little  better  how  the  process  works,  I  expected  to  gain  more  from  this 
workshop. 
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10.   Did  the  individual  sessions  accomplish  what  you      Yes  for  Yes  for  No  for 

expected?  Please  explain.  All:     21/75%  Some:  7/25%         All:  0/0% 

(9  comments) 


(3)    Yes  for  All.  We  accomplished  all  goals  and  developed  a  generic  HACCP  plan  that  is  very  helpful  to 
take  back  to  our  plants. 

(2)   Yes  for  All.  Material  was  covered  in  enough  depth  to  further  understanding  of  the  process.  The 
sessions  clarified  the  thought  processes  for  writing  a  HACCP  plan. 

(2)   Yes  for  All.  Great  group  discussions —  both  large  group  and  small  group —  good  mix/ variety.  Joint 
decisions  covered  most  aspects  of  a  subject. 

(1)   Yes  for  All  Some  of  the  sessions  were  too  stretched  out,  but  this  was  mainly  due  to  the  Participants 
themselves  and  not  to  the  facilitators. 

(1)   Some.  The  Subject  Matter  Expert  for  my  small  group  work  was  very  well-informed  and  a  helpful 
source  of  information.  A  lot  of  seemingly  unclear  situations  were  clarified. 


COMMENTS 

11.  Methods  other  than  a  workshop  format  could  have  been  employed  to  develop  a  generic  HACCP  model. 
Examples  of  options  include  an  FSIS  request  for  (a)  comments  on  an  FSIS-generated  model  published  in 
the  Federal  Register;  (b)  development  of  a  model  by  selected  academicians;  or  (c)  submission  to  FSIS  of 
independently-generated  models.  Please  indicate  your  preference  for  one  of  the  above-listed  options, 
or  describe  any  others  not  mentioned  that  you  would  like  to  be  considered  by  FSIS  in  the  future. 

(23  comments) 


Concerning  use  of  a  workshop  format: 

(8)  This  workshop  is  the  best  way.  I'm  not  in  favor  of  any  of  the  above  methods.  The  workshop  method 
was  most  effective:  direct  participation  was  an  integral  part  of  the  program  development  Industry 
and  FSIS  both  understand  the  problems  and  need  to  work  together  setting  up  programs  that  work 
for  industry  and  the  consumer.  Programs  like  this  should  not  be  dictated.  Input  from  bom  industry 
and  FSIS  is  the  best  option.  Methods  other  than  the  one  used  would  have  been  extremely 
time-consuming  and  likely  ineffective.  This  method  was  the  best  in  that  industry  had  enormous 
involvement  I  prefer  the  method  that  was  used  because  it  allowed  industry  personnel  to  submit  their 
views,  discuss  solutions,  and  solve  problems  with  the  help  and  cooperation  of  the  FSIS  HACCP  Special 
Team. 

(3)  I  like  the  workshop  format  This  was  the  best  possible  format  that  could  have  been  used.  The 
workshop  format  is  excellent  and  should  be  maintained  as  is. 

(1)  Independently-generated  models  may  aid  in  visual  comprehension  of  the  HACCP  plan.  Suggest 
independent  models  be  combined  with  the  workshop. 

Cc  icerning  other  options: 

(6)  Option  (C):  submission  to  FSIS  of  independently-generated  models. 

(2)  Option  (B):  development  of  a  model  by  selected  academicians. 

(1)  Option  (A):  comments  on  an  FSIS-generated  model  published  in  the  Federal  Register. 
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(1)   Concerning  seeking  comments  on  published  models,  as  in  Option  (A),  people  normally  don't  take  time 
to  comment. 


( 1 )    Get  more  opinions  through  mailing. 


12.  What  was  most  beneficial  to  you  about  the  workshop? 
(31  comments) 

(10)  Small  groups.  Breakout  sessions.  Working  in  the  small  groups  and  doing  only  small  segments  at  a 
single  time  was  most  beneficial.  One-on-one —  presentations  in  small  groups.  Small  groups  actually 
had  better  discussions  than  the  large  group  on  the  specifics  of  the  program.  More  came  out  in  the 
small  groups —  more  open  discussion.  Good  effort  by  facilitators  to  assist  group  members  in 
understanding  the  HACCP  plan.  Gave  us  better  participation  by  everybody. 

(5)   Working  with  other  people  in  the  industry.  Bringing  together  of  industry  representatives.  Working 
together  with  other  operations  people  and  feeling  that  we  have  a  part  in  developing  the  program. 
Interaction  of  industry  representatives  on  self-regulation;  interchange  of  ideas. 

(4)   Interaction  with  FSIS  personnel  and  Participants  from  industry.  Being  involved  with  the  whole 

development  process  rather  than  just  the  end  result.  Development  of  the  plan,  meeting  and  talking  to 
other  industry  members  as  well  as  USDA  officials.  Having  input  from  the  industry  is  the  best  step 
FSIS  could  have  made —  we  all  feel  we  have  had  input  into  it. 

(4)   Free-flowing  large  group  discussions  and  lots  of  brainstorming  allowed  many  viewpoints  to  be  heard. 
The  comments  made  by  FSIS  representatives  and  industry  Participants  during  the  many  discussions 
on  the  HACCP  model. 

(4)   A  clearer  understanding  of  the  HACCP  goals  and  of  the  process  involved  in  HACCP.  I  finally 

understand  the  principles  of  HACCP  and  the  use  of  the  decision  tree.  Learning  how  to  use  a  HACCP 
program. 

(2)  The  way  that  everyone  came  to  agreement  on  the  Critical  Control  Point  issues  even  after  some  heated 
discussion.  Input  could  be  made  even  if  it  was  not  accepted. 

(1 )  The  process  flow  diagram  was  set  up  ahead  of  time. 

(1 )  The  computer  overlays  and  printouts. 


13.  What  was  least  beneficial  to  you  about  the  workshop? 
(20  comments) 


(4)  Nothing.  Everything  was  beneficial. 

(3)  Lots  of  bickering  over  unimportant  issues,  but  I  guess  that  would  be  expected  in  large  groups.  When 
we  got  stuck  in  large  group  sessions  the  facilitator  should  have  moved  the  group  along.  FSIS  needs 
to  be  more  advisory  in  their  role. 

(2)  Redundancy  and  getting  bogged  down  in  interpretation  of  the  English  language.  A  lot  of  unnecessary 
repetition. 
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(2)    Lectures  were  boring.  Lecturers  were  ineffective  in  keeping  an  attentive  audience.  Though  the 
presenters  are  very  well-informed  on  the  material  they  present,  they  need  to  make  the  talks  more 
interesting.  Ifs  hard  for  an  audience  to  stay  focussed  on  straight-forward  talking  with  no  change  in 
partem.  Add  an  occasional  joke,  short  story. 

(2)  Explanations  of  principles.  The  long  dissertations  covering  book  matter. 

(1)  An  hour  or  so  should  have  been  devoted  at  the  start  of  the  workshop  to  better  explain  procedures  and 
goals. 

(1)  Some  instruction  or  explanation  concerning  the  reading  material  would  have  helped. 

(1)  Developing  the  work  stations  was  least  beneficial  because  I  think  they  need  to  be  plant-specific. 

(1 )  The  3-day  format;  workshop  could  be  completed  in  2  days. 

(1 )  The  large  amount  of  handouts  at  the  beginning  of  the  program. 

(1 )  Not  having  cokes  at  the  morning  breaks. 

(1)  The  location:  downtown  was  very  expensive  for  everything  from  parking  to  food,  etc. 


14.  Please  comment  on  the  methods  used  in  this  workshop  to:  (1)  assess  hazards  and  identify  preventive 
measures;  and  (2)  identify  CCPs  through  use  of  a  decision  tree. 

(30  comments) 

Concerning  (1) —  assessing  hazards  and  identifying  preventive  measures: 

(2)  Very  useful  as  far  as  identifying  hazards.  This  is  a  very  good  method  to  identify  and  assess  hazards. 

(2)  Large  group  discussions  were  great  for  these  two  elements.  Brought  out  many  different  ideas  from  a 
large  segment  of  the  pork  slaughter  industry  and  different  points  of  view. 

(1)  Well-explained. 

(1 )  Excessive  FSIS  involvement. 

Concerning  (2) —  identifying  CCPs  through  use  of  a  decision  tree: 

(4)  The  decision  tree  was  excellent  It  worked  well.  It  very  clearly  allowed  one  to  determine  a  Critical 
Control  Point  The  decision  tree  is  very  simple  but  effective. 

(3)  The  decision  tree  was  very  useful  in  establishing  a  protocol  for  identifying  CCPs.  It  is  a  structured 
format  that  minimizes  generalizations.  The  decision  tree  was  a  good  idea  that  highlighted 
involvement  and  repetition  of  some  CCPs,  and  thus  brought  about  changes  for  the  better. 

(1)  Great  use  of  computer  overheads  made  it  easy  to  follow  along  and  pinpoint  CCPs. 

(1)  The  Participants  maybe  were  a  little  too  eager  to  identify  some  of  the  CCPs,  but  the  method  was  great 

(1)  Acceptable. 

(1 )  Somewhat  confusing  at  times. 
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(1)    Is  more  work  and  not  as  much  fun,  but  something  that  has  to  be  discussed. 

Concerning  both  (1)  and  (2): 

(7)   Excellent.  Very  good  on  both.  Both  were  effective  methods.  Very  well-done.  I  think  that  enough  time 
(right  amount)  was  spent  on  it.  Enjoyed  all  the  input 

(3)   Facilitators  provided  too  little  direction  during  discussions,  which  made  sessions  run  very  long.  I 
understand  that  this  approach  was  intentional,  but  I  would  have  liked  to  have  seen  some  moderation 
of  this.  Example  models  should  have  been  presented.  FSIS  knew  what  they  wanted  out  of  this 
workshop  when  they  started.  More  detailed  guidelines  would  have  shortened  time  and  allowed 
conversations  on  other  vital  topics. 

(1)   Adequate  and  acceptable. 

(1)   The  methods  are  useful  to  aid  us  in  the  thought  process  to  accomplish  our  objectives. 


15.  Please  use  the  rest  of  this  page  to  make  any  additional  comments  that  may  help  us  understand  your 
thoughts  about  the  workshop  and  improve  future  activities. 

(18  comments) 


(4)   Very  good  workshop.  It  was  handled  very  well.  Thanks.  The  USDA  did  an  outstanding  job  of  hosting 
the  workshop.  It  was  well-organized  and  well-orchestrated.  All  of  the  facilitators  did  an  excellent  job 
of  helping  us  develop  a  model  without  dictating  or  staying  completely  noncommittal. 

(3)   The  workshop  was  a  good  forum  for  discussion.  Subject  matter  is  very  useful.  The  workshop  was  very 
useful  and  I  am  glad  to  see  industry  having  a  part  in  developing  a  workable  program. 

(2)   Experiences  from  previous  workshops  allowed  USDA  to  present  information  and  obtain  consensus 
easily.  USDA  did  an  excellent  job.  This  approach  should  be  used  more. 

(2)   FSIS's  use  of  visual  aids,  computers  and  microphones  was  very  impressive  and  added  greatly  to  the 
program  because  communication  was  made  very  easy. 

(2)   Could  have  cut  the  size  of  the  workbook  considerably  by  downsizing  the  material.  Too  much  paper 
was  wasted  in  making  copies  of  the  many  individual  overheads.  Could  an  outline  be  put  together  in 
12-point  letter-size  which  could  include  information  from  many  overheads? 

(1 )  Workshop  could  have  been  completed  in  2  days  rather  than  3. 

(1)  Perhaps  need  evening  socials  or  a  brief  get-together. 

(1)  We  must  make  the  generic  models  proactive  in  protection. 

(1 )  Very  professional  FSIS  personnel —  maybe  a  few  more  than  we  needed. 

(1)  The  workshop  discussion  seemed  to  be  dominated  by  a  few  industry  representatives. 
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SECTION  II—  OBSERVERS 


SUMMARY  OF  OBSERVERS'  RESPONSES 
EVALUATION  QUESTIONNAIRE 


u  orkshop  tor  HACC11  Model  I  development  for 
Swine  Slaughter  (Market  Hogs),  March  31  -  April  2, 1992 


SUMMARY   OF  OBSERVERS'   RESPONSES  : 
RESPOXDEXT  CHARACTERISTICS 

1.     Which  grouping  below  best  characlcrizcs  your  current  affiliation? 

Trade  Association:  1/50% 
Other  Government:  1/50% 


2.     If  you  marked  "Meat  &  Poultry  Industry"  in  Question  1,  please  indicate  all  groupings  below  which  best 
characterize  vour  industry  type  and  the  total  production  volume  of  your  establishment(s).  [Low  Volume 
Establishment  =  1  -  59,999  lbs.  produced/Quarter;  Medium  Volume  Establishment  =  60,000  -  1,000,000  lbs. 
produced /Quarter;  High  Volume  Establishment  =  more  than  1,000,001  lbs.  produced /Quarter] 


Not  applicable. 

3.  Days  attended  (mark  all  that  apply):  1  Day:  0/0%  2  Days:  0/0%  3  Days:  1/100%: 

4.  Please  check  one:  Observer:    2  Participant:  0 


HACCP  MODEL  DEVELOPMENT 


Very 

Successful 

Successful 

Somewhat 
Successful 

Not 

Successful 

Very  Don't 
Unsuccessful  Know 

Overall,  how  successful  was  the  workshop  in: 

a. 

Identifying  Critical  Control  Points  (CCPs)? 

1/50% 

1/50% 

0/0% 

0/0% 

0/0% 

0/0% 

b. 

Determining  critical  limits  for  each  CCP? 

0/0% 

2/100% 

0/0% 

0/0% 

0/0% 

0/0% 

c. 

Developing  monitoring  plan  activity  for 
identified  CCPs? 

1/50% 

1/50% 

0/0% 

0/0% 

0/0% 

0/0% 

d. 

Developing  corrective  action  approaches? 

0/0% 

2/100% 

0/0% 

0/0% 

0/0% 

0/0% 

e. 

Developing  record-keeping  systems? 

0/0% 

2/100% 

0/0% 

0/0% 

0/0% 

0/0% 

f. 

Developing  verification  procedures? 

0/0% 

2/100% 

0/0% 

0/0% 

0/0% 

0/0% 

g- 

Developing  a  generic  HACCP  model? 

1/50% 

1/50% 

0/0% 

0/0% 

0/0% 

0/0% 

NOTE:  Results  are  presented  as  number  of  responses/%  of  responses  for  each  category. 

1  Due  to  the  small  number  of  observers,  percentages  may  be  misleading. 

2  Only  1  of  the  2  respondents  answered  this  question. 
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DFS1GN  AND  OPrRATION  OF  WORKSHOP 
6.     Please  rate  the  workshop  in  terms  of: 

More  than 
Adequate 

Adequate 

Marginally 
Adequate 

Inadequate 

Very 

Inadequate 

Don't 
Know 

a. 

Development  of  a  generic  product /process 
Ho^rrintion 

1/50% 

1/50% 

0/0% 

0/0% 

0/0% 

0/0% 

b 

L^L  V          yji  1  It- 1 1 1  LU  L/IULLjj                   1U  Lil  cl  i  1  I 

2/100% 

0/0% 

0/0% 

0/0% 

0/0% 

0/0% 

r 

Orcpini 7A Hon  nf  thr*  fhrop  Hav? 

W  t  LI  CI  111  f  4  1  I  1  V^/  1  1   W 1     1 1  1  V.    1  1  11  ^V.    \-4  CI  \ 

0/0% 

2/100% 

0/0% 

0/0% 

0/0% 

0/0% 

d. 

Use  of  small  "break-out"  groups 

1/50% 

1/50% 

0/0% 

0/0% 

0/0% 

0/0% 

e. 

Reporting  of  small  "break-out"  group 
activities  to  entire  group 

0/0% 

1/100% 

0/0% 

0/0% 

0/0% 

0/0%  3 

f. 

Opportunity  for  input  from  Observers 

0/0% 

2/100% 

0/0% 

0/0% 

0/0% 

0/0% 

g- 

Opportunity  for  input  from  Participants 

1/50% 

1/50% 

0/0% 

0/0% 

0/0% 

0/0% 

h. 

Effectiveness  of  facilitators  in  helping 
achieve  workshop  objectives 

1/50% 

1/50% 

0/0% 

0/0% 

0/0% 

0/0% 

i. 

Effectiveness  of  subject  matter  experts  in 
helping  achieve  workshop  objectives 

0/0% 

2/100% 

0/0% 

0/0% 

0/0% 

0/0% 

j- 

Effectiveness  of  group  leaders  in  helping 
achieve  workshop  objectives 

1/50% 

1/50% 

0/0% 

0/0% 

0/0% 

0/0% 

k. 

Effectiveness  of  group  reporters  in  helping 
achieve  workshop  objectives 

1/50% 

0/0% 

0/0% 

0/0% 

0/0% 

1/50% 

1. 

Completion  of  workshop  objectives. 

1/50% 

1/50% 

0/0% 

0/0% 

0/0% 

0/0% 

7. 

Please  indicate  your  degree  of  agreement 
with  each  of  the  following  statements. 

Strongly 
Agree 

Agree 

Neutral 

Disagree 

Strongly 
Disagree 

Don't 
Know 

a. 

The  HACCP  model  developed  by  the 
workshop  participants  is  a  viable  frame- 
work for  the  industry  to  implement 
on  a  test  basis. 

U/U  /C 

11 OU  /O 

l/JU  /© 

U/U  IV 

U/U  /€> 

U/U  /c 

b. 

The  workshop  format  offers  a  sound 
approach  for  producing  a  generic  HACCP 
model  for  other  meat  and  poultry  products. 

0/0% 

2/100% 

0/0% 

0/0% 

0/0% 

0/0% 

c. 

The  presence  of  Observers  was  beneficial 
to  the  workshop. 

1/50% 

0/0% 

0/0% 

0/0% 

0/0% 

1/50% 

d. 

The  workshop  resulted  in  a  useful  generic 
HACCP  model  developed  by  industry 
participants. 

1/50% 

0/0% 

1/50% 

0/0% 

0/0% 

0/0% 

3 

Only  1  of  the  2  respondents  answered  this  question. 

-12- 


WORKSHOP  MATERIALS  (Observers:  Please  skip  Question  8  and  continue  with  Question  9.) 

Very  Somewhat      Not  Not  At  All 

Useful  Useful         Useful  Useful  Useful 

8.  Overall,  how  useful  were  the  written  materials  for: 

a.  Assessing  finished  product  hazards? 

b.  Assessing  ingredient  hazards? 

c.  Determining  placement  of  CCPs  on  flow 
diagram? 

d.  Determining  critical  limits  for  each  CCP? 

e.  Developing  assigned  CCP  monitoring  plan? 

f.  Developing  corrective  action  approaches? 

g.  Developing  record-keeping  systems? 

h.  Developing  verification  procedures? 

i.  Developing  a  generic  HACCP  model? 

WORKSHOP  EXPECTATIONS 

9.  Did  the  workshop  accomplish  what  you  expected?  Yes:  2/100%  No:  0/0% 
Please  explain. 

(1  comment) 

(1)    Yes.  The  workshop  completed  the  development  of  a  generic  model.  Only  once  it  is  implemented  will  it 
be  known  how  practical  the  model  is  to  use. 

10.  Did  the  individual  sessions  accomplish  what  you      Yes  for  Yes  for  No  for 
expected?  Please  explain.                                   All:     2/100%          Some:  0/0%  All:  0/0% 

No  comments. 

COMMENTS 

11.  Methods  other  than  a  workshop  format  could  have  been  employed  to  develop  a  generic  HACCP  model. 
Examples  of  options  include  an  FSIS  request  for  (a)  comments  on  an  FSIS-generated  model  published  in 
the  Federal  Register;  (b)  development  of  a  model  by  selected  academicians;  or  (c)  submission  to  FSIS  of 
independently-generated  models.  Please  indicate  your  preference  for  one  of  the  above-listed  options, 
or  describe  any  others  not  mentioned  that  you  would  like  to  be  considered  by  FSIS  in  the  future. 

(1  comment) 

Concerning  use  of  a  workshop  format: 

(1)   Perhaps  a  workshop  (industry/ government)  to  debate  and  compare  models  developed  by  all  three 
methods  listed  above  might  be  useful. 


OBSER\TRS  WERE  INSTRUCTED  TO 
SKIP  THIS  QUESTION. 
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Concerning  other  options: 


No  comments. 


12.  What  was  most  beneficial  to  you  about  the  workshop? 

(2  comments) 

0 )    Gathering  of  industry  and  FSIS  on  a  mutual  basis  and  letting  industry  take  the  lead  in  developing  a 
model  plan. 

(1 )    Observing  the  debate  between  the  various  Participants. 

13.  What  was  least  beneficial  to  you  about  the  workshop? 


No  comments. 


14.   Please  comment  on  the  methods  used  in  this  workshop  to:  (1)  assess  hazards  and  identify  preventive 
measures;  and  (2)  identify  CCPs  through  use  of  a  decision  tree. 


Concerning  (1) —  assessing  hazards  and  identifying  preventive  measures: 


No  comments. 


Concerning  (2) —  identifying  CCPs  through  use  of  a  decision  tree: 


No  comments. 


Concerning  both  (1)  and  (2): 
(1  comment) 


(1)  These  methods  produced  a  result.  The  usefulness  of  the  end  product  will  not  be  known  until  it  is 
implemented  but  there  doesn't  seem  to  be  another  viable  alternative  for  development. 


15.  Please  use  the  rest  of  this  page  to  make  any  additional  comments  that  may  help  us  understand  your 
thoughts  about  the  workshop  and  improve  future  activities. 


No  comments. 
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HACCP 

Workshops  Report 


Overview  of  the  five 
Workshop  Steering  Committee 

Reports 


In  January  1990,  the  Food  Safety  and  Inspection  Service  (FSIS)  of 
the  U.S.  Department  of  Agriculture  initiated  efforts  to  determine 
how  to  implement  the  Hazard  Analysis  and  Critical  Control  Point 
(HACCP)  system  in  regulated  meat  and  poultry  plants.  HACCP 
is  a  process  control  system  that  identifies  "critical"  points  in  the 
food  production  process  that  should  be  controlled  to  prevent  food 
safety  hazards  (i.e.,  microbiological,  chemical,  and  physical 
hazards)  from  occurring. 

Agency  activities  included  consultations  and  public  hearings  to 
explain  HACCP  and  the  Agency's  efforts,  workshops  to  develop 
model  HACCP  plans,  in-plant  testing  to  trial  the  specific  plant 
HACCP  plans  at  nine  volunteer  plants,  and  an  assessment  of  the 
overall  experience  of  the  Agency. 

A  series  of  reports  will  be  written  on  the  various  phases  and 
activities  undertaken  by  the  Agency  and  participants  during  the 
study.  This  is  one  of  the  reports.  For  further  information 
concerning  FSIS  HACCP  activities  contact: 

Dr.  Dorothy  Stringfellow 
HACCP  Office 

14th  &  Independence  Ave.,  SW 
Room  4444 

Washington,  DC  20250 
Phone:  (202)  690-2087 
FAX:  (202)  690-3886 


HACCP  WORKSHOPS  REPORT 


Overview  of  the  five 
WORKSHOP  STEERING  COMMITTEE 
REPORTS 


Vol.  Ill 


United  States  Department  of  Agriculture 
Food  Safety  and  Inspection  Service 
Policy  Analysis  Unit 
Washington,  D.C.  20250 
February  1994 


SUMMARY 


The  Food  Safety  and  Inspection  Service  (FSIS),  USDA,  initiated  a  study  to  determine  how  to  implement 
HACCP*  into  meat  and  poultry  inspection  operations.  As  part  of  the  study,  five  workshops  were  held 
with  industry  to  develop  five  model  HACCP  plans  for  specific  products  and  processes.  The  five 
products/processes  chosen  were  Refrigerated  Foods,  Cooked  Sausage,  Poultry  Slaughter,  Ground  Beef,  and 
Swine  Slaughter.  Three  of  the  generic  models  developed  at  the  workshops  (Refrigerated  Foods,  Cooked 
Sausage,  and  Poultry  Slaughter)  were  tested  in  three  volunteer  plants  each  for  a  total  of  nine  pilot  plants. 

At  the  completion  of  each  workshop,  a  steering  committee  of  industry  members  was  chosen  to  fine-tune 
the  generic  model  developed  at  the  workshop.  In  addition,  the  committee  would  represent  industry  in 
altering  the  generic  model  if  needed  after  the  development  and  testing  of  plant-specific  HACCP  plans  in 
the  pilot  plants. 

An  examination  of  the  five  steering  committees  showed  that  the  committees  played  an  important  role  by 
fine-tuning  the  HACCP  models  and  ensuring  their  consistency. 

Steering  committee  members  said  industry  representatives  were  needed  to  complete  any  tasks  left 
unfinished  at  the  workshops. 

Steering  committee  members  said  industry  representatives  were  needed  as  communication  links 
between  industry  and  FSIS  staff  involved  with  HACCP  implementation. 

Steering  committee  members  said  industry  representatives  needed  to  be  responsible  for  adding  (or 
deleting)  critical  control  points  (CCPs)  as  necessary  after  a  generic  model  developed  at  a 
workshop  had  been  matched  to  plant-specific  HACCP  plans. 

Steering  committee  members  said  industry  members  wanted  to  know  that  industry  representatives 
were  responsible  for  "wordsmithing"  the  generic  industry  models  developed  at  the  workshops  if 
they  were  going  to  "claim  ownership"  of  the  models. 

One  outcome  of  the  development  of  HACCP  steering  committees  was  increased  communication  among 
industry  members  and  an  increase  in  HACCP  related  activities  by  the  committee  members'  companies. 


*  Hazard  Analysis  Critical  Control  Points  (HACCP)  is  a  process  control  system  that  identifies  "critical"  points  in  the  food  production 
process  that  should  be  controlled  to  prevent  food  safety  hazards  from  occurring. 
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I.  INTRODUCTION 


Evaluation 

To  describe  the  role  of  steering  committees  as  part  of  the  workshop  method  for  developing  generic  model 
HACCP*  plans  for  meat  and  poultry  products. 


Background 

In  January  1990,  the  Food  Safety  and  Inspection  Service  (FSIS)  began  the  HACCP  Implementation  Study 
to  determine  how  to  implement  Hazard  Analysis  Critical  Control  Points  (HACCP)  into  meat  and  poultry 
inspection  operations.  As  part  of  the  study,  five  workshops  were  held  with  industry  to  develop  model 
HACCP  plans  for  specific  products  and  processes.  Those  chosen  included  Refrigerated  Foods,  Cooked 
Sausage,  Poultry  Slaughter,  Ground  Beef,  and  Swine  Slaughter.  Three  of  the  generic  models  developed 
at  the  workshops  (Refrigerated  Foods,  Cooked  Sausage,  and  Poultry  Slaughter)  were  tested  in  three 
volunteer  plants  each  for  a  total  of  nine  pilot  plants. 

Steering  Committees 

A  steering  committee  of  five  industry  representatives  was  established  at  the  completion  of  each  of  the  five 
HACCP  workshops.  The  representatives  were  chosen  from  the  working  groups  that  had  been  formed 
during  the  workshops.  Each  steering  committee  selected  a  chairperson  and  sometimes  a  co-chairperson 
to  facilitate  the  transactions  of  the  committee. 

Steering  committees  served  three  functions: 

•  Fine-tuned  the  model  HACCP  plan  for  that  particular  product/process  category  for  consistency 
and  readability. 

•  Addressed  any  unresolved  issues  from  the  workshop. 

•  Represented  industry  in  altering  the  generic  model  if  it  needed  to  be  changed  following 
development  of  plant-specific  plans  in  the  volunteer  plants. 

Special  Team 

The  Special  Team,  consisting  of  seven  FSIS  staff  from  headquarters  and  field,  was  responsible  for  the 
implementation  of  HACCP  Study  activities  and  data  collection  activities  including  workshops,  in-plant 
training,  and  pilot  plant  tests.  If  the  Special  Team  determined  after  working  with  the  pilot  plants  that  the 
generic  model  developed  at  a  workshop  needed  to  be  changed,  then  the  Special  Team  would  notify  the 
steering  committee  chairperson  for  that  product  so  the  committee  could  reconvene  to  discuss  the  problem. 
Dr.  Nathaniel  Clark  of  the  HACCP  Special  Team  was  steering  committee  coordinator  and  served  as 
interface  between  the  steering  committees,  the  Special  Team,  and  other  FSIS  HACCP  activities.  He  gave 
the  committees  their  charge  from  the  workshops  and  explained  their  duties.  The  committees  sent  their 
completed  models  through  him  to  the  Special  Team. 


*  Hazard  Analysis  Critical  Control  Points  (HACCP)  is  process  control  system  that  identifies  "critical"  points  in  the  food  production 
process  that  should  be  controlled  to  prevent  food  safety  hazards  from  occurring. 


Methodology 

The  Policy  Analysis  Unit  (PAU),  FSIS,  asked  steering  committee  members  about  the  role  of  HACCI 
steering  committees  as  part  of  an  examination  of  the  HACCP  workshops. 

Three  questionnaires  were  developed  to  address  steering  committee  activities  over  the  life  of  the 
committees.  Interview  questions  addressed  topics  concerning  the  roles  and  tasks  of  the  steering  committee, 
the  amount  of  time  and  effort  spent  by  committee  members,  and  direct  costs  incurred  by  industry  for  theii 
representatives  on  the  committee. 

The  first  questionnaire  focused  on  steering  committee  activities  during  the  HACCP  workshop. 

•    The  second  questionnaire  was  designed  to  collect  information  on  steering  committee  activities 
for  the  period  of  time  between  the  HACCP  workshop  and  the  pilot  plant  on-site  visits. 

The  third  questionnaire  requested  information  on  steering  committee  activities  during  and 
after  the  pilot  plant  on-site  visits.  It  also  addressed  the  role  of  the  steering  committee  int 
general. 

The  questionnaires  were  administered  either  in  person  or  by  telephone  to  the  FSIS  steering  committee! 
coordinator,  steering  committee  chairpersons,  and  steering  committee  members.  Interviews  with  the: 
steering  committee  chairperson  and  members  from  each  workshop  took  place  after  their  generic  model  was 
finalized.  The  steering  committee  coordinator  was  interviewed  after  each  workshop  and  periodically  to 
determine  the  status  of  the  various  committees,  if  they  had  completed  fine-tuning  their  model  or  if 
additional  interviews  with  the  steering  committees  would  be  needed,  depending  on  whether  generic  models! 
had  to  be  changed  following  implementation  in  the  pilot  plants. 


Report  Plan 

First,  this  report  presents  a  description  of  each  of  the  HACCP  steering  committees  in  the  following  five 
chapters.  The  final  chapter  provides  a  summary  and  the  committees'  recommendations  for  future  steering 
committee  activities.  Responses  to  the  three  questionnaires  administered  to  each  committee  are  found  in 
the  Appendices. 


2 


II.  REFRIGERATED  FOODS  STEERING  COMMITTEE* 


Introduction 

The  Refrigerated  Foods  Workshop  was  held  in  Baltimore,  Maryland,  on  February  26-28,  1991.  The 
particular  process  under  consideration  was  cooked  and  assembled  products.  At  the  completion  of  the 
workshop  after  the  generic  model  had  been  completed,  the  following  persons  were  chosen  by  the 
workshop  participants  to  represent  them  on  the  steering  committee: 

Martin  Mitchell,  Chairman  Salad  Mfg.  Association 

Mike  Ensley  Tyson  Foods,  Inc. 

Margaret  Gerwin  Stouffer  Foods  (Nestle  USA) 

John  Humber  Kraft  General  Foods,  Inc. 

Edward  Krysinski  Campbell  Soup  Company 

The  Refrigerated  Foods  Steering  Committee  was  charged  with  the  following  responsibilities: 

1.  Fine-tune  the  DRAFT  generic  model  developed  at  the  workshop. 

2.  Address  an  issue  raised  in  the  workshop  on  whether  "seeding"  (spike  verification)  should  be 
identified  as  a  verification  procedure  in  the  HACCP  model.  ["Seeding"  involves  sending  a  product 
with  a  known  microbiological  count  through  a  process  to  see  if  the  controls  in  place  will  find  it] 

3.  Address  whether  "receiving"  should  be  a  critical  control  point  (CCP). 

4.  Address  an  issue  raised  regarding  the  applicability  of  HACCP  Principle  #1  concerning  Risk 
Assessment  and  Risk  Categories.  Workshop  participants  had  questioned  the  value  of  assigning 
a  risk  category  to  ingredients  and  finished  product 


Steering  Committee  Activities 

The  Refrigerated  Foods  Steering  Committee  held  an  organizational  meeting  at  the  end  of  the  workshop. 
At  that  time  they  selected  a  chairman  and  assigned  tasks  to  members.  The  committee  met  in  Washington, 
D.C.  on  March  19,  1991  to  polish  the  DRAFT  generic  model  and  discuss  the  other  assigned  tasks.  The 
completed  generic  model  was  sent  to  the  HACCP  Special  Team  on  April  25,  1991. 

The  issue  of  "seeding"  (spike  verification)  was  resolved  by  retaining  it  in  verification  procedures  under 
CCP  #7,  assembling  components  into  a  package.  This  was  not  considered  a  substantive  change  to  the 
generic  plan  by  the  HACCP  Special  Team. 

Initially,  the  issue  of  whether  "receiving"  should  be  a  CCP  was  resolved  by  the  committee  adding  a 
paragraph  at  the  end  of  the  flow  diagram  of  the  plant's  process  stating  their  position  that  all  HACCP  plans 
should  consider  including  "receiving"  as  a  CCP  depending  on  the  ingredient(s)  and  its  suppliers.  In 
November  1991,  the  HACCP  Steering  Committee  Coordinator  asked  the  steering  committee  to  reconsider 
"receiving"  as  a  CCP  after  all  three  pilot  plants  included  it  in  their  plant-specific  plans. 

*See  Appendix  A  for  more  detailed  information. 
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The  steering  committee  discussion  on  the  applicability  of  HACCP  #1  concerning  Risk  Assessment  and 
Risk  Categories  resulted  in  a  communication  through  the  HACCP  Special  Team  to  the  National  Advisory 
Committee  on  Microbiological  Criteria  For  Food  (NACMCF).  Since  that  time,  NACMCF  has  rewritten 
Principle  #1  using  a  decision  tree  method  to  determine  hazard  levels. 


Findings 

The  findings  from  the  questionnaires  administered  to  the  committee  members  and  coordinator  supported 
the  notion  that  the  steering  committee  played  a  necessary  role  fine-tuning  the  generic  model  so  that  it  was 
a  consistent,  unified  document.  Committee  members  also  stated  that  as  an  industry  group,  they  developed 
a  generic  model  more  suitable  to  industry. 

They  expressed  that  the  expense  of  being  a  part  of  the  steering  committee  was  not  a  burden  to  their 
companies  although  they  were  concerned  that  it  might  be  to  smaller  companies.  Their  companies  paid 
the  expenses  for  the  Washington  meeting,  telephone,  and  fax  costs.  Their  companies  were  very  supportive 
of  their  involvement. 

Members  of  this  committee  were  the  only  ones  to  express  a  concern  that  there  was  no  FSIS  representative: 
on  the  committee. 
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II.  COOKED  SAUSAGE  STEERING  COMMITTEE* 


Introduction 


The  Cooked  Sausage  Workshop  was  held  in  Fort  Worth,  Texas,  on  May  21-23,  1991.  At  the  completion 
of  the  Cooked  Sausage  Workshop,  the  following  persons  were  chosen  by  the  workshop  participants  to 
represent  them  on  the  steering  committee: 


The  Cooked  Sausage  Steering  Committee  was  charged  with  fine-tuning  the  DRAFT  generic  model 
developed  at  the  workshop.  No  unresolved  issues  were  raised  at  the  workshop. 

Steering  Committee  Activities 

The  Cooked  Sausage  Steering  Committee  did  not  meet  separately  at  the  workshop  to  choose  a  chairperson 
or  delegate  assignments.  Workshop  participants  had  indicated  a  desire  that  they  all  be  present  when  the 
duties  of  the  steering  committee  and  the  questionnaire  were  presented.  Steering  committee  members  were 
introduced  and  William  Sveum  volunteered  to  serve  as  chairman.  A  PAU  analyst  presented  a  synopsis 
of  the  questionnaire  to  the  entire  workshop  at  the  request  of  the  HACCP  Special  Team  Leader  and  the 
workshop  participants. 

The  committee  held  four  conference  calls  to  fine-tune  the  generic  model.  Originally,  the  committee 
wanted  to  revise  the  generic  model  developed  at  the  workshop.  Members  stated  it  contained  too  many 
CCPs  and  several  sanitation  issues  had  been  left  unresolved.  The  Director  of  the  HACCP  Special  Team 
told  committee  members  they  could  only  fine-tune  the  workshop-generated  model.  However,  they  could 
develop  and  submit  an  alternative  model.  Committee  members  decided  to  develop  an  alternative  model 
which  they  prepared  during  their  conference  calls. 

The  committee  submitted  the  fine-tuned  workshop  model  and  the  alternative  model  to  the  HACCP  Special 
Team  on  September  5,  1991.  The  Special  Team  forwarded  the  alternative  model  to  the  HACCP  office 
for  review. 


The  findings  from  the  questionnaires  administered  to  the  committee  members  and  coordinator  supported 
the  notion  that  steering  committees  were  an  essential  part  of  the  workshop  process.  They  were  most 
useful  to  industry  in  making  the  wording  of  the  HACCP  model  uniform  to  avoid  misinterpretations  and 
ensure  consistency. 

*See  Appendix  B  for  detailed  information. 


William  H.  Sveum,  Chairman 
Dr.  Mohammad  F.  Azam 
J.  Patrick  Hadden 
Kathleen  L.  Hanigan 
Mike  Robach 


Oscar  Mayer 
Thorn  Apple  Valley,  Inc. 
Webb  Technical  Group 
Farmland  Foods,  Inc. 
Continental  Grain  Co. 


Findings 
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However,  committee  members  also  were  frustrated  because  they  thought  that  as  workshop  representatives 
they  should  be  able  to  make  substantive  changes  in  the  generic  model.  They  complained  that  their  rol 
was  limited  by  only  being  able  to  "wordsmith"  the  model.  They  would  have  liked  the  role  of  th< 
committee  to  be  expanded  to  handle  issues  they  felt  were  unresolved. 

None  stated  that  their  participation  had  entailed  much  cost  to  their  companies.  The  committee'! 
companies  paid  for  the  conference  calls,  faxes,  and  other  telephone  calls.  Members  spent  about  16  hour 
either  working  individually  on  their  assignments  or  participating  on  conference  calls.  Their  companies 
were  supportive  of  their  involvement. 
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IV.  POULTRY  SLAUGHTER  STEERING  COMMITTEE* 


Introduction 


The  Poultry  Slaughter  HACCP  Workshop  was  held  at  Atlanta,  Georgia,  on  August  27-29,  1991.  At  the 
completion  of  the  workshop,  the  following  persons  were  chosen  by  the  workshop  participants  to  represent 
them  on  the  steering  committee: 


The  Poultry  Slaughter  Workshop  Committee  was  charged  with  fine-tuning  the  DRAFT  generic  model 
developed  at  the  workshop.  In  addition,  "notes  from  industry"  attached  to  the  generic  model  concerning 
chemical  residues  and  chlorination  were  to  be  addressed  by  the  committee. 


Steering  Committee  Activities 

The  Poultry  Slaughter  Steering  Committee  held  a  brief  organizational  meeting  at  the  conclusion  of  the 
workshop.   Mike  Gregory  was  selected  chairman. 

The  committee  did  not  meet  again  as  a  committee  either  in  person  or  via  conference  call.  Each  member 
was  assigned  one  of  the  columns  in  the  generic  model  such  as  "Critical  Limits"  to  fine-tune.  When  their 
corrections  were  completed,  members  faxed  them  to  the  chairperson  who  assembled  their  drafts  and 
returned  the  unified  document  for  comments.  The  final  draft  was  submitted  to  the  HACCP  Steering 
Committee  Coordinator  on  November  15,  1991. 

The  steering  committee  chairman  addressed  the  "notes  from  industry."  In  his  submittal  letter  to  FSIS,  he 
acknowledged  the  notes  and  endorsed  them  as  recommendations  to  FSIS.  The  chemical  residue  note  was 
an  acknowledgement  that  the  poultry  industry  already  has  in  place  effective  preventative  and  monitoring 
programs  in  the  live  production  area.  Inclusion  of  chemical  residues  as  a  CCP  at  slaughter  would  be  a 
duplication  of  effort  and  less  effective.  The  chlorination  note  was  a  series  of  recommended  check  points 
to  be  considered  as  the  CCP's  regarding  chlorination  of  the  chillers  are  implemented  during  the  pilot  plant 
studies. 


The  findings  from  the  questionnaires  administrated  to  the  committee  members  and  coordinator  supported 
the  notion  that  it  was  important  for  industry  to  participate  on  the  steering  committee. 


♦See  Appendix  C  for  detailed  information. 


James  L.  Ayres 
Leo  C.  McKee 
Steve  Mitchell 
Norbert  D.  Neal 


Mike  Gregory,  Chairman 


Hudson  Foods  Inc. 
Gold  Kist  Inc. 
O.K.  Foods,  Inc. 
Con  Agra  Broiler  Co. 
Louis  Rich  Co. 


Findings 


Members  spent  about  four  to  twelve  hours  on  their  assignments.  Cost  to  their  companies  was  in  lost  hours! 
and  cost  of  telephone  calls  and  faxes.  Their  companies  were  supportive  and  were  beginning  to  use 
HACCP  principles  in  their  plants. 

Most  members  noted  their  role  was  to  fine-tune  the  generic  model  and  adhere  strictly  to  the  desires  of  the 
workshop  participants.  On  the  other  hand,  one  member  felt  restricted  by  that  role  and  would  have  liked 
to  have  been  able  to  make  alternations  to  the  model. 

Members  were  happy  with  the  method  they  used,  i.e.  being  assigned  columns  to  "wordsmith  model.  The 
chairman,  however,  thought  that  it  might  have  been  better  if  he  had  gotten  the  group  together  for  a 
meeting  or  at  least  a  conference  call  to  shorten  the  process. 
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V.  GROUND  BEEF  STEERING  COMMITTEE* 


Introduction 


The  Ground  Beef  HACCP  Workshop  was  held  in  Phoenix,  Arizona,  on  December  3-5,  1991.  At  the 
completion  of  the  workshop,  the  following  persons  were  chosen  by  the  workshop  participants  to  represent 
them  on  the  steering  committee: 


The  Ground  Beef  Workshop  Committee  was  charged  with  fine-tuning  the  DRAFT  generic  model 
developed  at  the  workshop.  No  unresolved  issues  were  raised  at  the  workshop. 

Steering  Committee  Activities 

The  Ground  Beef  Steering  Committee  held  an  organizational  meeting  at  the  completion  of  the  workshop. 
A  committee  chairman  and  co-chairman  were  selected  and  assignments  were  made.  Committee  members 
worked  on  their  assignments  and  faxed  their  changes  to  the  chairperson.  The  committee  held  a  conference 
call  on  December  19, 1991,  to  review  the  changes  they  had  made  and  faxed  to  the  chairman.  He  prepared 
a  draft  and  sent  members  a  final  copy  for  review. 

The  committee  submitted  its  model  to  the  HACCP  Special  Team  dated  December  19,  1991.  Since  no 
ground  beef  processes  were  pilot  tested  using  this  model,  the  committee's  work  was  essentially  completed 
within  a  few  weeks. 


The  findings  from  the  questionnaires  administered  to  the  committee  members  and  coordinator  supported 
the  notion  that  the  chairman  stressed  the  need  to  complete  steering  committee  activities  promptly. 
Therefore,  committee  members  felt  pressured  to  get  the  job  done.  They  worked  efficiently  on  their  own 
and  cooperated  well  as  a  team.  They  stated  that  they  had  benefitted  by  the  work  of  previous  committees. 

Members  spent  from  two  to  eight  hours  on  steering  committee  activities.  None  felt  it  was  a  financial 
burden  to  their  companies  as  the  only  expenses  were  telephone  calls  and  faxes.  Their  companies  were 
supportive  of  their  participation.  In  fact,  three  members  said  their  companies  either  would  like  to  have 
been  a  pilot  plant,  or  were  embarking  on  a  HACCP  project 


♦See  Appendix  D  for  detailed  information. 


Ryan  Englund,  Chairman 
William  R.  Lloyd,  Co-Chairman 
Kofi  Boaitey 
Karen  Procheniewski 
John  Southerland 


Jac  Pac  Foods 
Monfort,  Inc. 


Carlkarcher  Enterprises,  Inc. 
Wis-Pak  Foods 
Portion-trol  Foods,  Inc. 


Findings 
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VI.  SWINE  SLAUGHTER  STEERING  COMMITTEE* 


Introduction 


The  Swine  Slaughter  HACCP  Workshop  was  held  in  Minneapolis,  Minnesota,  on  March  31-  April  2, 
1992.  At  the  completion  of  the  workshop,  the  following  persons  were  chosen  by  the  workshop 
participants  to  represent  them  on  the  steering  committee: 


The  Swine  Slaughter  Workshop  Committee  was  charged  with  fine-tuning  the  DRAFT  generic  model 
developed  at  the  workshop.  Committee  members  were  asked  to  address  as  addenda  to  the  model  issues 
of  pesticides  and  antibiotic  residues  and  also  organic  acid  sanitizing  systems  for  microbial  reductioa 

Steering  Committee  Activities 

The  Swine  Slaughter  Steering  Committee  held  a  brief  organizational  meeting  at  the  completion  of  the 
workshop.  A  committee  chair  and  co-chair  were  selected.  One  committee  member  volunteered  to  write 
the  addenda  for  the  committee  to  address  during  their  conference  calls.  The  chairman  set  up  a  conference 
call  for  April  22, 1992,  during  which  the  committee  discussed  each  step  and  voted  on  any  changes.  Then, 
the  chair  faxed  members  a  draft  of  the  changes  which  they  discussed  at  another  conference  call  on  May 
6, 1992.  Shortly  afterwards,  the  chair  submitted  their  model  to  the  HACCP  Special  Team,  dated  May  7, 
1992,  and  the  addenda  written  by  Kirk  Jones  and  reviewed  by  the  committee. 

The  addendum  on  pesticides  and  antibiotic  residues  urged  cooperation  among  regulatory  agencies,  swine 
producers  and  packers  to  reduce  the  chemical  residue  hazard  and  listed  ways  to  accomplish  the  reduction. 
The  addendum  on  organic  acid  sanitizing  systems  suggested  that  although  these  systems  were  no  substitute 
for  Good  Manufacturing  Practices,  they  would  be  useful  as  a  means  of  preventing  bacterial  growth  during 
and  after  the  slaughter  process. 

On  June  12,  1992,  the  Steering  Committee  Coordinator  informed  the  chair  that  some  footnotes  about 
operational  sanitation  needed  to  be  incorporated  into  the  model.  The  chair  did  this  and  notified  the  steering 
committee  of  her  action.  The  committee's  work  was  then  completed  as  there  were  no  swine  slaughter 
pilot  plants  using  this  model. 


The  findings  from  the  questionnaires  administered  to  the  committee  members  and  coordinator  supported 
the  notion  that  the  committee's  role  was  to  fine-tune  the  generic  model  for  consistency  and  to  eliminate 
any  wording  that  would  indicate  a  specific  method  of  control  must  be  used  at  a  CCP. 

'"See  Appendix  E  for  detailed  information. 


Kathleen  L.  Hanigan,  Chair 
Ellen  L.  Graber,  Co-Chair 
Paul  T.  Conner 
Kirk  Jones 
Bob  Neubauer 


Farmland  Foods,  Inc. 
Thorn  Apple  Valley 
IBP 

Monfort,  Inc.-Pork  Division 
Johnsonville  Foods 


Findings 
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Members  spent  from  three  to  twenty  hours  working  on  steering  committee  business.  The  only  expense 
to  their  companies  other  than  lost  employee  hours  was  the  cost  of  the  conference  call,  faxes,  and  other 
telephone  calls.  Their  companies  were  supportive  of  their  participation.  At  least  one  company  plans  to 
develop  its  own  HACCP  plan. 

One  member  suggested  that  the  steering  committee  should  become  a  standing  committee  which  would  act 
as  a  forum  for  dealing  with  industry-wide  issues. 
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VH.  SUMMARY 


Introduction 

Industry  steering  committees  were  established  after  each  HACCP  workshop  to  refine  the  generic  model 
created  at  the  workshop,  to  make  any  changes  in  the  model  following  testing  in  the  pilot  plants,  and  to 
address  any  unresolved  issues  from  the  workshops.  The  committee  reported  to  the  Special  Team 
coordinator  who  acted  as  liaison  between  the  committee  and  the  HACCP  office  and  sent  a  letter  thanking 
the  committee  when  their  work  was  completed. 

This  chapter  summarizes  the  activities  of  all  the  steering  committees.  Topics  covered  include  the 
composition  of  the  committees,  cost  to  industry  of  steering  committee  participation,  steering  committee 
roles,  and  steering  committee  operatioa  Finally,  the  chapter  provides  comments  and  makes 
recommendations  based  on  the  findings. 


Steering  Committee  Composition 

Working  groups  at  each  workshop  chose  one  of  their  members  to  serve  on  the  steering  committee  for  that 
product/process.  The  HACCP  steering  committee  coordinator  had  suggested  to  participants  that  they  select 
only  industry  participants  for  steering  committee  membership.  The  composition  of  steering  committees 
differed  from  workshop  to  workshop.  Some  had  more  large  companies  represented  and  some  had  more 
small  companies;  some  had  representatives  from  labs  and  some  had  trade  association  representatives.  There 
were  no  FSIS  representatives. 

The  mix  of  companies  did  not  seem  to  affect  steering  committee  work.  More  important  than  the  company 
size  appeared  to  be  the  characteristics  of  the  individuals  selected.  Industry  people  who  attended  the 
workshop  were  knowledgeable  about  the  actual  processes  in  the  plants  and  made  interested  and  active 
committee  members.  Key  to  steering  committee  performance  was  the  effectiveness  of  the  steering 
committee  chairperson. 

Cost  to  Industry  of  Steering  Committee  Participation 

Most  of  the  cost  to  industry  was  in  lost  working  hours  of  steering  committee  members.  Members 
estimated  they  had  spent  two  to  twenty  hours  on  steering  committee  business.  The  committee  chairpersons 
spent  up  to  40  hours.  Companies  of  steering  committee  members  also  incurred  the  cost  of  secretarial  time 
and  incidental  costs  such  as  conference  calls  and  faxes. 

Committee  members  were  encouraged  by  their  companies  to  participate  on  steering  committee  activities. 
Most  committee  members  remarked  that  their  companies  were  glad  to  have  them  participate  so  that  the 
company  could  be  involved  in  HACCP  from  the  beginning. 
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Steering  Committee  Roles 
Fine  tuning  the  model 

Most  steering  committees  perceived  their  role  as  fine  tuning  or  "wordsmithing"  the  generic  model  to  be 
consistent  within  the  model  and  across  the  other  generic  models.  "Wordsmithing"  the  model  was  the 
major  role  all  the  committees  played.  Members  thought  FSIS  would  have  fine-tuned  the  model  differently. 
Also,  they  thought  that  industry  would  be  more  likely  to  accept  the  model  knowing  that  industry 
representatives  had  completed  it. 

The  Cooked  Sausage  Steering  Committee  members  initially  thought  their  role  allowed  them  to  modify  the 
generic  model  developed  at  the  workshop  at  their  discretion.  However,  they  were  discouraged  from 
making  substantive  changes  to  the  model  by  the  Director  of  the  HACCP  Special  Team.  At  subsequent 
workshops,  the  steering  committee  coordinator  was  careful  to  stress  to  steering  committee  members  that 
their  role  was  to  "wordsmith"  the  model.  He  told  the  committees  that  if  substantial  changes  needed  to 
be  made  in  the  generic  model,  the  HACCP  Special  Team  would  notify  them,  but  they  were  not  to  initiate) 
changes  on  their  own. 

Changing  the  generic  model  during/after  the  on-sites 

In  order  for  a  generic  model  to  be  modified,  the  HACCP  Special  Team  needed  to  make  a  request  to  the 
steering  committee  for  that  product/process.  Changes  to  the  generic  model  were  based  on  what  happened 
during  on-site  visits  to  the  pilot  plants  and  afterwards  when  plant-specific  plans  were  matched  to  the 
generic  model  developed  at  the  workshops. 

The  Refrigerated  Foods  Steering  Committee  was  the  only  committee  asked  to  develop  new  CCPs  after  the 
development  of  plant-specific  plans.  The  HACCP  Special  Team  requested  that  the  committee  develop  new 
CCPs  for  "receiving"  as  all  three  pilot  plants  had  included  "receiving"  as  a  CCP  for  meat  and  non-meat 
ingredients  in  their  plant-specific  plans. 

• 

Other  tasks 


Participants  at  three  workshops  asked  their  steering  committees  to  perform  other  tasks  which  could  not 
be  completed  at  the  workshop.  All  these  committees  made  recommendations  about  their  tasks  and  referred 
them  to  the  HACCP  Special  Team  for  action. 

The  Refrigerated  Foods  Steering  Committee  was  charged  with  addressing  the  applicability  of  HACCP 
Principle  #1  and  whether  "seeding"  should  be  identified  as  a  verification  procedure.  The  committee  made 
recommendations  about  HACCP  Principle  #1  which  were  forwarded  to  the  National  Advisory  Committee 
on  Microbiological  Criteria  for  Food  (NACMCF).  "Seeding"  was  retained  in  the  model. 

The  Poultry  Slaughter  Steering  Committee  was  asked  to  look  at  several  "notes  to  industry"  added  to  the 
general  model  developed  at  the  workshop.  The  committee  approved  both  the  "chemical  residue"  and  the 
"chlorination"  notes  as  they  appeared  in  the  generic  model  developed  at  the  workshop. 

The  Swine  Slaughter  Steering  Committee  was  asked  to  attach  addenda  at  the  end  of  the  generic  model 
concerning  pesticides  and  antibiotic  residues  and  organic  acid  sanitizing  systems  for  microbial  reduction. 
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One  committee  member  prepared  the  addenda  which  the  committee  revised,  approved,  and  sent  to  the 
Special  Team. 

Steering  Committee  Operation 

Steering  committees  were  not  provided  with  specific  operating  procedures.  Each  committee  developed 
its  own  Most  committees  began  with  a  short  organizational  meeting  at  the  end  of  the  workshop.  The 
Refrigerated  Foods  Committee  held  a  one-day  meeting  in  Washington,  D.C.,  to  complete  its  tasks.  The 
Cooked  Sausage,  Ground  Beef,  and  Swine  Slaughter  Steering  Committees  used  conference  calls  to  perform 
their  work.  The  Poultry  Slaughter  Committee  members  worked  individually  and  the  chairperson  compiled 
their  work.  In  all  cases,  the  chairperson  would  contact  the  steering  committee  coordinator  if  the  committee 
had  questions  or  to  submit  the  finished  model.  Committees  took  between  two  weeks  and  three  and  1/2 
months  to  complete  "wordsmithing"  their  generic  model. 

The  steering  committee  coordinator  thought  the  work  of  a  steering  committee  might  be  expedited  by 
holding  a  committee  meeting.  However,  to  him  what  seemed  most  important  to  getting  the  task  of 
refining  the  model  done  was  a  good  chairperson  who  could  make  assignments  and  follow  up  on  tasks. 
For  the  final  two  steering  committees  the  coordinator  requested  that  a  co-chairperson  be  selected  as  he  had 
had  difficulties  sometimes  reaching  the  chairperson  of  some  committees. 


Recommendations  and  Comments 

According  to  steering  committee  members,  if,  in  fact,  a  workshop  format  will  be  used  in  the  future  to 
develop  HACCP  generic  plans,  then  steering  committees  will  be  necessary  adjuncts.  They  felt  steering 
committees  would  serve  a  number  of  important  functions  for  the  HACCP  workshops: 

•  Industry  members  will  want  to  know  that  industry  representatives  are  responsible  for 
"wordsmithing"  the  generic  industry  models  developed  at  the  workshops  if  they  are  going  to 
"claim  ownership"  of  the  models. 

•  Industry  representatives  will  be  needed  to  complete  any  tasks  left  unfinished  at  the 
workshops. 

•  Industry  representatives  will  be  needed  as  communication  links  between  industry  and  FSIS 
staff  involved  with  HACCP  implementation. 

•  Industry  representatives  will  need  to  be  responsible  for  adding  (or  deleting)  CCPs  as 
necessary  after  a  generic  model  developed  at  a  workshop  has  been  matched  to  plant-specific 
HACCP  plans. 

The  financial  burden  to  companies  was  generally  minimal.  It  consisted  of  minimal  lost  employee  time 
and  very  little  direct  financial  outlay.  Companies  with  steering  committee  members  appeared  pleased  at 
the  opportunity  to  be  a  part  of  the  HACCP  development  process. 
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Steering  committee  members  made  the  following  suggestions  for  alternative  methods  of  developing 
HACCP  plans  (see  HACCP  workshop  evaluation  report  for  additional  suggestions  made  by  workshop 
participants): 

•    FSIS  should  design  HACCP  models  and  place  them  in  the  Federal  Register  for  comment. 


For  products  for  which  many  Partial  Quality  Control  (PQQ  plans  have  been  written, 
generalize  from  these  plans  to  develop  a  HACCP  model. 

Designate  an  industry  steering  committee  for  each  product/process  to  design  a  HACCP  model. 


A  serendipitous  outcome  of  the  steering  committees  has  been  increased  communication  among  industry 
members.  Steering  committee  members  have  formed  informal  networks  of  contacts  to  consult  when  they 
have  questions  about  HACCP  or  related  matters.  One  member  even  suggested  that  the  committees  be  i 
turned  into  standing  committees  to  be  a  forum  for  industry  issues.  Others  felt  the  steering  committees : 
should  have  the  authority  to  develop  HACCP  plans. 

Also,  committee  members,  by  their  participation  on  the  steering  committee,  have  encouraged  increased 
HACCP  related  activities  in  their  companies.  A  number  of  the  companies  of  steering  committee  members 1 
are  developing  their  own  HACCP  plans  based  on  the  generic  models. 
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APPENDIX  A 


REFRIGERATED  FOODS  STEERING  COMMITTEE 
RESPONSES  TO  QUESTIONNAIRE 


L 


Policy  Analysis  Unit 
February  1994 


REFRIGERATED  FOODS 


Steering  Committee  Roles,  and  Time,  Effort,  and  Costs  Expended 

Questionnaire  I:  Steering  Committee  Meetings  during  the  HACCP  Workshop 

This  questionnaire  focuses  on  the  Steering  Committee's  activities  during  the  HACCP  Model 
Development  Workshop  at  Baltimore,  Md,  February  26-28, 1991.  A  Policy  Analysis  Unit  (PAU) 
analyst  recorded  information  at  the  time  of  the  workshop  meeting.  A  PAU  analyst  also  inter- 
viewed the  FSIS  Steering  Commitee  Coordinator,  Steering  Committee  Chairperson,  and  other 
committee  members. 


Q-l.  How  many  times  did  the  Steering  Committee  meet? 
Met  once  at  the  end  of  the  Baltimore  workshop. 

Q-2.  Please  estimate  the  number  of  hours  spent  relative  to  each  of  these  meetings  concerning  actual  time  in 
meetings,  as  well  as  any  preparation  time  for  the  meetings. 

Time  in  Meetings         Preparation  Time 

Meeting  2/28/91  30  -60  minutes  none 

Q-3.  In  general,  what  was  the  purpose  of  each  meeting?  What  topics  were  covered?  What  was  accomplished  in 
each  meeting?  If  the(se)  meetings  resulted  in  any  decisions,  or  written  documentation,  report(s)  or  other 
products,  please  specify  and  estimate  the  number  of  hours  expended  in  these  activities  that  were  not 
included  in  the  response  to  Question  2  above. 

The  Refrigerated  Foods  Steering  Committee  met  on  the  last  day  of  the  three-day  workshop.  After 
introductions  had  been  made,  the  Steering  Committee's  role  was  explained  by  Dr.  Nathaniel  Clark, 
FSIS  Steering  Committee  Coordinator,  and  the  chairperson,  Martin  Mitchell,  Salad  Mfg.  Association, 
was  selected.  The  committee  discussed  their  charges  and  assigned  tasks  to  committee  members.  The 
tasks  were  assigned  to  members  according  to  which  issues  individual  members'  working  groups  had 
discussed  at  the  Baltimore  workshop. 

Committee  charges  and  assignments: 

1.  Fine-tune  the  generic  HACCP  model.  Margaret  Gerwin,  Stouffer  Foods 

2.  Address  whether  "seeding"  should  be  a  verification  procedure.  Mike  Ensley,  Tysons  Foods 

3.  Address  HACCP  Principle  #1.  John  Humber,  Kraft  General  Foods,  Inc. 

In  addition,  the  chairperson  was  to  address  whether  "receiving"  should  be  a  critical  control  point  (CCP). 

The  committee  decided  to  meet  in  Washington,  D.C. ,  shortly  after  the  workshop  to  continue  their  work. 

Initially,  FSIS  staff  were  going  to  meet  with  the  Committee,  however  this  was  changed  when 

it  appeared  that  the  Coordinator,  Dr.  Clark;  HACCP  Special  Team  Chair,  Dr.  Leary;  a  HACCP  subject 

matter  expert;  and  a  union  representative  would  have  to  attend  if  any  FSIS  person  attended.  Thus, 

a  "mini  workshop"  would  have  resulted  and  FSIS  thought  it  best  to  have  the  Steering  Committee 

members  meet  alone.  The  committee,  however,  would  have  liked  to  have  had  FSIS  input  at  this 

meeting. 


Q-4.  Were  there  any  other  Steering  Committee  activities  that  occurred  during  the  workshop  that  haven't 
been  addressed?  If  yes,  please  explain. 

There  were  no  other  activities  during  the  workshop. 
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Steering  Committee  Roles,  and  Time,  Effort,  and  Costs  Expended 

OuesUonnaire  II:  Steering  Committee  Contacts  between  the  Workshop  and  the 

On-Site  Visit 

This  questionnaire  was  designed  to  collect  information  on  the  activities  of  the  Steering  Comm 
tee  for  the  period  of  time  between  the  HACCP  Workshop  and  the  Pilot  Plant  On-Site  Visit.  A' 
PAU  Analyst  administered  this  questionnaire  by  telephone  to  the  FSIS  Steering  Committee 
Coordinator,  the  Steering  Committee  Chairperson,  and  members  of  the  Steering  Committeee. 

Q-l.  What  activities  were  undertaken  by  the  Steering  Committee  during  the  time  period  between  the 

Workshop  and  the  On-Site  Visit  (e.g.,  meetings,  phone  calls,  etc.)?  What  topics  were  covered  and  what 
was  accomplished  by  each  activity?  How  much  time  was  spent  in  each  activity?  Was  there  preparation 
time  involved?  If  so,  how  much  time?  Overall,  how  significant  was  each  issue  that  was  addressed? 

Committee  members  prepared  individual  assignments  from  the  workshop  meeting  and  circulated 
drafts  before  their  next  meeting  to  be  held  in  Washington,  D.  C.  Each  member  spent  about  4-6  hours 
preparing  his/her  assignment 

The  committee  met  in  Washington,  D.C.  on  March  19, 1991,  for  4-5  hours  to  discuss  unresolved  issues 
from  the  Baltimore  workshop.  Each  committee  member  had  had  a  specific  "homework"  problem  to 
research  and  bring  to  this  meeting.  By  the  end  of  the  meeting,  all  members  agreed  in  principle  on  thi 
decisions  about  all  issues  and  all  that  was  left  for  the  individal  members  to  do  was  to  type  up  the 
committee's  decisions  in  final  form. 


Q-3. 


About  2 1/2  hours  were  spent  polishing  the  model.  This  was  lead  by  Margaret  Gerwin  who  was 
assigned  this  task  at  the  first  Steering  Committee  meeting  in  Baltimore.  She  had  reviewed  the 
model  as  developed  at  the  workshop  and  identified  the  areas/paragraphs  that  needed  to  be  adjusted 
to  finalize  the  model.  She  also  brought  to  the  D.C.  meeting  her  suggested  wording  based  on  thesa 
research  to  develop  standardization  of  terminology. 
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About  an  hour  was  spent  on  "seeding"  (spike  verification)  and  "receiving"  as  a  CCP.  Spike  verificatic 
was  retained  in  verification  procedures  under  CCP  #7 ,  assembling  components  into  a  package. 
"Seeding"  (spike  verification)  was  not  a  substantive  issue.  It  had  been  assigned  to  the  Steering 
Committee  in  order  to  stop  lengthy  unnecessary  discussion  by  a  few  workshop  participants  who  did : 
represent  the  opinions  of  most  participants  at  the  Baltimore  workshop.  The  decision  on  "receiving" 
left  at  a  compromise,  in  that  the  committee  added  a  paragraph  at  the  end  of  the  process  flow  diagram 
stating  their  position  that  all  HACCP  plans  should  consider  including  "receiving"  as  a  CCP  depend! 
on  prior  experience  with  the  ingredients)  and  its  supplier.  This  was  not  a  substantive  change  to  the 
generic  HACCP  plan. 

About  an  hour  was  spent  on  Principle  #1.  These  were  the  most  significant  deliberations  of  the  Ste< 
Committee.  They  resulted  in  a  communication  to  the  Special  Team  which  was  forwarded  to  FSIS 
Administrator,  Dr.  Lester  Crawford.  He  passed  their  concerns  on  to  the  National  Advisory  Committee 
On  Microbiological  Criteria  For  Food  (NACMCF).  The  NACMCF  is  currently  considering  changing 
Principle  #1  of  HACCP  Principles  For  Food  Production.  None  of  the  other  Steering  Comittee 
deliberations  suggested  substantial  changes  in  the  generic  HACCP  model. 

After  the  meeting,  committee  members  put  documents  into  final  form.  All  documents  had  several 
revisions  that  were  reviewed  by  members  via  fax  and  individual  telephone  calls. 

Q-2.  Was  the  entire  Steering  Committee  involved  in  these  activities  (meetings,  telephone  calls,  report  writing, 
etc.)?  Please  elaborate.  Yes   No   Both  X 


All  members  were  actively  involved  in  Steering  Committee  activities.  One  member  who  was  on  travel 
status  during  the  period  did  not  have  a  specific  assignment,  but  participated  in  the  group  meetings. 
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All  participated  at  the  meeting  in  D.C.  Otherwise,  the  chairperson  coordinated  on  an  individual  basis 
with  all  members  and  with  the  Special  Team  Steering  Committee  Coordinator. 

Q-3.  Was  anyone  other  than  Steering  Committee  members  involved  in  these  activities? 

Yes    X      No  (Continue  with  Q-4) 

If  yes,  please  elaborate. 

Steering  Committee  members  involved  their  staff  and  other  participants  from  the  Baltimore  workshop 
in  their  assignments.  M.  Gerwin's  secretary  was  involved  in  polishing  the  model.  J.  H umber  met  with 
one  or  two  colleagues  at  Kraft  foods  who  had  been  at  the  workshop  to  help  him  revise  Principle  #1. 
M.  Ensley  contacted  all  members  of  his  working  group  from  Baltimore  to  discuss  spike  verification. 


QA.  Were  there  any  other  Steering  Committee  activities  that  occurred  during  this  time  period  that  we  haven't 
discussed?  Yes  X  No  

If  yes,  please  explain. 

The  chairperson  reviewed  and  consolidated  individual  reports  from  committee  members,  wrote  a 
report  of  the  Steering  Committee's  proposed  recommendations  and  sent  it  to  the  Steering  Committee 
Coordinator.  This  report  was  reviewed  by  all  Special  Team  members  upon  receipt  On  April  25, 1991, 
the  Steeling  Committee  Coordinator  wrote  the  Steering  Committee  chairperson  stating  that  the  report 
had  been  accepted  except  for  four  areas  where  minor  changes  needed  to  be  incorporated.  Revisions 
were  made  as  requested  and  the  final  report  dated  April  25, 1991,  was  re-submitted  to  the  Special  Team 
by  the  chairperson. 

The  Special  Team  accepted  the  committee's  final  report  recommendations.  However,  because  the 
Special  Team  believed  that  the  Principle  #1  changes  proposed  by  the  Steering  Committee  were  beyond 
the  scope  of  the  HACCP  Special  Team's  responsibilities,  and  conflicted  with  their  application  of 
NACMCFs  HACCP  definitions  in  the  Agency's  HACCP  program,  this  matter  was  referred  to 
Dr.  Crawford,  FSIS  Administrator,  with  the  request  that  the  Steering  Committee's  proposal  be 
submitted  to  NACMCF  for  deliberation. 
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Steering  Committee  Roles,  and  Time,  Effort,  and  Costs  Expended 

Questionnaire  III:  Steering  Committee  Contacts  during  and  after  the  On-Site  Visit 

This  questionnaire  requested  information  on  the  Steering  Committee's  activities  during  and 
after  the  pilot  plant  On-Site  Visit.  It  also  addressed  the  role  of  the  Steering  Committee  in 
general.  This  instrument  was  adminstered  by  the  Policy  Analysis  Unit  by  telephone  to  the  FS 
Steering  Committee  Coordinator,  Assistant  Coordinator,  and  all  Steering  Committee  member 
upon  completion  of  the  On-Site  Evaluation  or  after  all  (if  any)  refinements/changes  to  the 
Generic  HACCP  Model  have  been  incorporated. 


Steering  Committee  Activities 


Q-l.  What  activities  were  undertaken  by  the  Steering  Committee  during  and  after  the  On-Site  Visit 

(e.g.,  meetings,  phone  calls,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics  were  covered  and 
what  was  accomplished  by  each  activity?  How  much  time  was  spent  in  each  activity?  Was  there 
preparation  time  involved?  If  so,  how  much  time?  [N.B.:  If  no  activities  occurred,  skip  to  Q-4J 
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On  November  12, 1991,  the  FSIS  Steering  Committee  Coordinator  submitted  a  request  from  the  HAC 
Special  Team  to  the  Steering  Committee  to  include  additional  CCFs  for  "receiving"  of  meat  and/or  non 
meat  ingredients.  The  Special  Team  made  this  request  because  all  Refrigerated  Foods  pilot  plants 
included  "receiving"  of  meat  and/or  non  meat  ingredients  as  CCFs  in  their  plant-specific  HACCP  plana 
The  Steering  Committee  Coordinator  has  contacted  the  Steering  Committee  chairperson  several  times 
since  the  initial  request,  but  to  date  there  has  been  no  word  of  committee  action. 


Q-2.  Was  the  entire  Steering  Committee  involved  in  these  activities  (meetings,  telephone  calls,  report  writing, 
etc.)?  Please  elaborate.  Yes   No  X 


Q-3.  Was  anyone  other  than  Steering  Committee  members  involved  in  these  activities? 

Yes   No  X      (Continue  with 

If  yes,  please  elaborate. 


Q-4.  Were  there  any  other  Steering  Committee  activities  that  occurred  during  this  time  period  that  we  haven't 
discussed?  Yes   No  X 


If  yes,  please  explain. 
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Steering  Committee  Role 


Q-5.  Now  that  you  have  served  as  a  member  of  the  Steering  Committee,  what  do  you  think  is  the  role  of  the 
Steering  Committee? 

"We  served  as  representatives  of  the  industry  people  at  the  workshop  and  finalized  the  generic  model 
in  a  manner  differently  than  USDA  would  have  done  it." 

"We  harmonized  the  generic  HACCP  model  so  that  it  looked  like  a  unified  document  rather  than  a 
patchwork  affair  put  together  by  several  committees." 

"We  resolved  issues  which  would  have  been  difficult  and  time  consuming  for  the  whole  workshop 
to  consider." 


a,  3-6.  In  brief,  what  did  you  do  to  perform  your  role? 

"As  chairperson,  I  served  as  a  "conduit"  of  information  to  and  from  the  FSIS  Steering  Committee 
Coordinator  rather  than  as  a  "repository"  of  information." 

"As  a  committee  member,  I  contacted  other  people  from  my  working  group  at  Baltimore  to  help  me 
prepare  my  assignment  for  the  Washington  meeting.  I  contacted  other  committee  members  by 
phone  and  fax.  I  was  an  active  participant  at  the  Washington  meeting. " 


3-7.  Please  note  any  other  comments  you  would  like  to  offer  concerning  the  role,  time  and  effort  of  the 
Steering  Committee.  Consider  usefulness  of  the  Committee  in  polishing  and/or  refining  the  Generic 
HACCP  Model,  any  benefits  to  you  personally  and /or  your  company,  travel  expenses,  costs  of 
telephone  calls,  etc. 

Overall,  committee  members  seemed  pleased  to  be  involved  in  the  Steering  Committee,  even  though 
some  activities  took  them  longer  to  achieve  man  they  initially  expected.  Committee  members  were 
happy  to  be  a  part  of  the  decisionmaking  process. 

Steering  Committee  members  said  that  the  time  and  expense  of  being  a  part  of  the  committee  was  not 
a  burden  although  two  member  felt  that  it  could  be  a  hardship  for  a  person  coming  from  a  small 
company. 

"I  work  with  USDA  and  FDA  most  every  day  and  I  was  impressed  from  Day  One  [with  this  program]. 
This  was  the  best  run,  best  managed  program.  USDA  should  tip  their  hats  and  take  a  bow.  All  of  the 
committee  members  feel  this  way  and  so  do  a  lot  of  the  people  I  spoke  with  at  the  Baltimore  meeting. 
We're  positively  impressed.  USDA  is  doing  a  great  job.  If  I  could  suggest  one  thing  to  be  done 
differently,  it  would  be  to  have  the  materials  given  out  in  advance  of  the  workshop  so  the  participants 
could  have  come  better  prepared." 

"I  feel  very  good  about  having  been  on  the  Steering  Committee  and  having  shared  ideas,  feelings  and  a 
spirit  about  this  activity  and  about  what  we  accomplished.  There  is  a  pride  or  egotism  in  having  been 
involved  in  such  an  activity  from  the  beginning  on  the  ground  floor  and  in  having  the  ability  to 
influence  policy.  I  was  impressed.  Everyone  stayed  on  target" 

"The  negative  aspects  of  being  on  the  Steering  Committee  were  the  time  spent  away  from  normal  work 
duties  and  the  dollar  expenses  involved.  It  would  have  been  difficult  for  someone  from  a  small 
company  to  have  been  on  the  Steering  Committee  because  of  the  commitment  of  time  and  money. 
All  of  us  were  from  large  companies  (corporations)  except  me  (the  Chairperson)  and  I'm  from  an 
association  that  represents  smaller  companies.  Therefore,  I  felt  I  was  the  one  on  the  committee  who 
best  presented  the  interests  of  the  small  companies.  Because  I'm  Technical  Director  of  Salad 
Manufacturers,  they  undewrote  fifty  percent  of  my  expenses." 


REFRIGERATED  FOODS 
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Q-8.  What  are  your  ideas  concerning  the  usefulness  and  viability  of  the  function  of  the  Steering  Committee? 

"USDA  couldn't  have  done  this  [develop  a  HACCP  modell  without  the  Steering  Committee.  If  they 
had  done  it  themselves,  they  would  have  polished  the  model  differently.  This  was  the  best  way  to 
develop  an  industry  model." 

"The  model  produced  at  the  workshop  was  good,  but  unpolished,  and  it  was  appropriate  to  have  a  sm; 
group  (i.e.,  the  Steering  Committee)  fix  it  up." 

Committee  members  felt  strongly  that  it  was  important  to  get  good  people  [to  serve  on  steering 
committees].  This  [Refrigerated  Foods]  Steering  Committee  was  composed  of  people  who  had  experie 
with  the  actual  process.  Steering  committees  should  be  composed  of  such  people  rather  that 
"academics"  who  have  no  experience  preparing  the  product  at  a  plant 

Several  committee  members  expressed  a  concern  that  there  was  not  a  USDA  representative  at  the 
Washington  meeting.  Although  they  understood  the  need  to  draw  up  an  industry  HACCP  model, 
an  FSIS  person  there  would  have  been  helpful  for  answering  questions.  On  the  other  hand,  one 
member  said  that  it  was  probably  good  that  FSIS  wasn't  involved.  The  Steering  Committee 
Coordinator  said  it  would  be  best  if  a  HACCP  Special  Team  member  could  be  mere,  however,  current  i 
regulations  would  not  allow  only  one  FSIS  representative. 


The  Steering  Committee  was  needed  to  handle  issues  that  were  unresolved  at  the  conclusion  of  the 
workshop.  Committee  members  were  gratified  that  their  handling  of  Principle  #1  was  taken 
seriously  and  referred  on  to  the  NACMCF. 
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Steering  Committee  Role  and  Time,  Effort,  and  Costs  Expended 

Questionnaire  I:  Steering  Committee  Meetings  during  the  HACCP  Workshop 

This  questionnaire  focused  on  the  Steering  Committee's  activities  during  the  HACCP  Model 
Development  Workshop  at  Ft.  Worth,  TX,  on  May  21-23, 1991.  A  Policy  Analysis  Unit  (PAU) 
Analyst  recorded  information  at  the  time  of  the  workshop  meeting.  A  PAU  Analyst  also  inter- 
viewed the  FSIS  Steering  Commiteee  Coordinator,  the  Steering  Committee  Chairperson,  and 
other  committee  members. 


Q-l.  How  many  times  did  the  Steering  Committee  meet? 

The  Cooked  Sausage  Steering  Committee  did  not  meet  separately  at  the  Ft  Worth  meeting.  Workshop 
participants  wanted  everyone,  not  just  committee  members,  to  hear  the  charge  to  the  committee  and 
the  intent  of  the  evaluation  questionnaire. 


Q-2.  Please  estimate  the  number  of  hours  spent  relative  to  each  of  these  meetings  concerning  actual  time  in 
meetings,  as  well  as  any  preparation  time  for  the  meetings. 

N/A 

Q-3.  In  general,  what  was  the  purpose  of  each  meeting?  What  topics  were  covered?  What  was  accomplished  in 
each  meeting?  If  the(se)  meetings  resulted  in  any  decisions,  or  written  documentation,  report(s)  or  other 
products,  please  specify  and  estimate  the  number  of  hours  expended  in  these  activities  that  were  not 
included  in  the  response  to  Question  2  above. 

On  May  23, 1991,  at  the  end  of  the  working  group  sessions,  when  the  full  group  reconvened,  Special 
Team  members  introduced  the  participants  who  had  been  elected  to  the  Steering  Committee  from  each 
working  group. 

Dr.  Nathaniel  Clark,  FSIS  Steering  Committee  Coordinator,  reiterated  the  role  of  the  Steering  Committee 
to  the  workshop  participants.  He  stated  a  chairperson  needed  to  be  elected  and  asked  if  there  were  any 
volunteers.  William  Sveum,  Oscar  Mayer,  volunteered  and  was  unanimously  accepted  by  the  committee 
members. 

The  committee  was  not  charged  with  any  tasks  other  than  "wordsmithing"  the  model  developed 
at  the  workshop. 

Q-4.  Were  there  any  other  Steering  Committee  activities  that  occurred  during  the  workshop  that  haven't 
been  addressed?  If  yes,  please  explain. 

After  the  session  was  dismissed,  a  PAU  analyst  introduced  herself  to  the  Steering  Committee 
chairperson  and  said  she  or  another  analyst  from  PAU  would  be  in  touch  with  him  and  the 
other  committee  members  after  the  generic  model  was  fine-tuned. 
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Steering  Committee  Role  and  Time,  Effort,  and  Costs  Expended 

Questionnaire  II:  Steering  Committee  Contacts  between  the  Workshop  and  the 

On-Site  Visit 

This  questionnaire  was  designed  to  collect  information  on  the  activities  of  the  Steering  Comir 
tee  for  the  period  of  time  between  the  HACCP  Workshop  and  the  Pilot  Plant  On-Site  Evalua- 
tion. A  PAU  Analyst  administered  this  questionnaire  by  telephone  to  the  FSIS  Steering  Comrri 
tee  Coordinator,  the  Steering  Committee  Chairperson,  and  members  of  the  Steering  Committel 

Q-l.  What  activities  were  undertaken  by  the  Steering  Committee  during  the  time  period  between  the  Works 
and  the  On-Site  Evaluation  (e.g.,  meetings,  phone  calls,  etc.)?  What  was  the  purpose  of  each  activity? 
What  topics  were  covered  and  what  was  accomplished  by  each  activity?  How  much  time  was  spent  in 
each  activity?  Was  there  preparation  time  involved?  If  so,  how  much  time? 

The  committee  did  not  hold  a  meeting.  It  accomplished  its  task  via  four  conference  calls 
(7/3,7/19,7/24,  and  8/23)  each  approximately  two  hours  in  length  at  which  time  members  went  over  the  i 
model  developed  at  the  workshop  step  by  step,  "wordsmithing"  it  for  consistency  with  itself  and  with 
the  Refrigerated  Foods  HACCP  model  The  chairperson  coordinated  the  conference  calls,  faxed 
updated  drafts  to  members,  and  acted  as  facilitator.  In  addition,  individual  phone  calls  were  made 
between  members. 

Originally,  the  committee  wanted  to  be  able  to  revise  the  generic  model  developed  at  the  workshop. 
Members  felt  there  were  too  many  CCPs  in  the  generic  model  and  that  a  couple  of  issues  regarding 
sanitation  had  been  left  hanging.  The  Special  Team  Leader  told  committee  members  they  could  only 
polish  the  workshop  model  but  they  could  develop  and  submit  an  alternative  HACCP  model  to  FSIS. 
The  chairperson  suggested  to  the  committee  that  each  member  write  an  alternative  generic  model.  An 
alternative  model,  developed  from  committee  suggestions,  was  discussed  and  polished  by  committee 
members  during  their  conference  calls. 

The  committee  submitted  the  polished  workshop  model  to  the  HACCP  Special  Team  on 
September  5, 1991.  In  addition,  it  presented  the  alternative  model  which  it  had  developed.  The 
committee  would  like  to  see  the  alternative  model  presented  to  the  industry.  To  date  no  action  has 
been  taken  on  this  request 

Q-2.  Was  the  entire  Steering  Committee  involved  in  these  activities  (meetings,  telephone  calls,  report  writing, 
etc.)?  Please  elaborate.  Yes  X  No  


Steering  Committee  members  worked  together  and  came  to  decisions  by  consensus  during  the 
conference  calls. 

Q-3.  Was  anyone  other  than  Steering  Committee  members  involved  in  these  activities? 

Yes  X  No  (Continue  with  Q-4 

If  yes,  please  elaborate. 

The  chairperson's  secretary  would  retype  the  drafts  and  fax  them  to  committee  members. 
Committee  members  used  colleagues  in  their  organizations  for  specific  information  about 
various  processes. 

Q-4.  Were  there  any  other  Steering  Committee  activities  that  occurred  during  this  time  period  that  we  haven't 
discussed?  Yes   No  X 


If  yes,  please  explain. 

There  were  no  other  activities  during  this  period. 
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Q-l.  What  activities  were  undertaken  by  the  Steering  Committee  during  and  after  the  On-Site  Evaluation 

{e.g.,  meetings,  phone  calls,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics  were  covered  and 
what  was  accomplished  by  each  activity?  How  much  time  was  spent  in  each  activity?  Was  there 
preparation  time  involved?  If  so,  how  much  time?  [N.B.:  If  no  activities  occurred,  skip  to  Q-4.] 
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Steering  Committee  Role  and  Time,.  Effort,  and  Costs  Expended 

Questionnaire  III:  Steering  Committee  Contacts  during  and  after  the  On-Site  Visit 

This  questionnaire  requested  information  on  the  Steering  Committee's  activities  during  and 
after  the  pilot  plant  On-Site  Evaluation.  It  also  addressed  the  role  of  the  Steering  Committee  in 
general  This  instrument  was  adrninstered  by  the  Policy  Analysis  Unit  by  telephone  to  the  FSIS 
Steering  Committee  Coordinator  and  to  all  Steering  Committee  members  upon  completion  of 
the  On-Site  Evaluation  or  after  all  (if  any)  refinements/changes  to  the  Generic  HACCP  Model 
had  been  incorporated. 


Steering  Committee  Activities 


The  committee  had  no  activities  during  the  On-Site  Evaluation  period. 


Q-2.  Was  the  entire  Steering  Committee  involved  in  these  activities  (meetings,  telephone  calls,  report  writing, 
etc.)?  Please  elaborate.  Yes   No 

N/A 


Q-3.  Was  anyone  other  than  Steering  Committee  members  involved  in  these  activities? 

Yes   No  (Continue  with  Q-4) 

If  yes,  please  elaborate. 


N/A 

Q-4.  Were  there  any  other  Steering  Committee  activities  that  occurred  during  this  time  period  that  we  haven't 
discussed?  Yes   No  , 

If  yes,  please  explain. 

N/A 


Steering  Committee  Role 


Q-5.  Now  that  you  have  served  as  a  member  of  the  Steering  Committee,  what  do  you  think  is  the  role  of  the 
Steering  Committee? 

According  to  committee  members,  the  role  of  the  Steering  Committee  was  to  polish  the  generic  model 
However,  they  felt  this  role  was  limiting.  "All  it  [the  committee]  can  do  is  wordsmith,  it  can't  really 
clean  up  the  model."  It  was  disheartening  that  we  could  not  make  changes.  The  Steering  Committee 
should  be  permitted  to  change  CCPs.  It  has  no  clout  at  all."  They  had  felt  that  as  a  steering  committee 
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elected  by  members  of  the  workshop,  they  could  represent  workshop  participants  and  make  changes 
in  the  model 

They  felt  their  role  should  be  expanded  to  handle  unresolved  issues.  'In  the  workshop  certain  issues 
were  not  aired  because  of  the  presence  of  the  press.  In  a  small  group  like  the  Steering  Committee, 
issues  could  be  discussed  more  confidentially/' 

"Because  we  were  a  small  group  and  were  all  technical  and  quality  control  people  who  understood  the 
process,  we  could  do  a  better  job  of  determining  a  HACCP  plan."  "Some  people  [at  the  workshop]  wet 
interested  only  in  production." 


Q-6.  In  brief,  what  did  you  do  to  perform  your  role? 

The  chairperson  coordinated  the  conference  calls,  took  notes,  wrote  down  ideas,  and  drew  up  drafts 
which  he  faxed  to  members. 

Committee  members  wrote  alternative  HACCP  plans  and  participated  in  conference  calls.  One  put  the 
drafts  on  his  portable  computer  so  that  he  could  keep  up-to-date  with  changes  to  the  model. 


Q-7.  Please  note  any  other  comments  you  would  like  to  offer  concerning  the  role,  time  and  effort  of  the 
Steering  Committee.  Consider  usefulness  of  the  Committee  in  polishing  and/or  refining  the  Generic 
HACCP  Model,  any  benefits  to  you  personally  and /or  your  company,  travel  expenses,  costs  of 
telephone  calls,  etc. 

The  chairperson  said  he  had  spent  40  hours  on  activities  for  the  Steering  Committee.  Other  members! 
estimated  they  had  spent  8  hours  in  addition  to  the  conference  calls  for  a  total  of  approximately  16  how 
None  felt  their  participation  had  entailed  much  cost  to  their  companies  particularly  since  the  group 
did  not  meet  and  therefore  had  encurred  no  travel  expenses.  Members  had  either  volunteered  to  be 
on  the  Steering  Committee  or  did  not  turn  down  the  nomination  because  they  felt  it  was  a  plus  for  then 
and  their  companies.  Their  companies  were  glad  to  be  involved  in  developing  what  they  considered  an 
important  document 

One  member  who  was  QC  director  at  his  plant  planned  to  take  advantage  of  his  involvement  in  HACCP 
and  start  a  program  at  his  plant  Another  who  worked  for  a  lab  hoped  mat  some  day  his  participation 
would  benefit  his  company.  "As  a  result  of  their  involvement,  they  [his  company]  would  be  able  to 
help  others  with  HACCP  plans  and  would  understand  where  USD  A  and  industry  are  coming  from." 


Q-8.  What  are  your  ideas  concerning  the  usefulness  and  viability  of  the  function  of  the  Steering  Committee? 

"It  [the  Steering  Committee]  is  most  useful  to  industry  in  making  the  wording  [of  a  HACCP  generic 
plan]  uniform  so  that  there  are  no  mixed  interpretations  of  what  it  means." 

"The  Steering  Committee  is  absolutely  essential  to  make  the  generic  plan  more  understandable  to 
industry  and  the  wording  consistent  It  is  necessary  to  look  at  what  was  done  by  previous  steering 
committees  so  the  models  read  the  same.  Committees  need  to  contact  chairpersons  of  the  other 
steering  committees.  The  plan  should  be  totally  understandable  so  industry  can  pick  up  plans  and 
write  their  own  specific  plans  from  the  model." 

"Given  its  limitations,  it  was  useful  in  providing  an  alternative  model." 

"I'm  in  favor  of  steering  committees.  It  [the  Cooked  Sausage  Steering  Committee]  worked  effectively 
without  meeting  because  most  members  were  QC  people  and  technical  people." 

"The  key  was  to  do  it  as  a  group,  not  divide  areas;  to  come  to  consensus  and  work  from  that  point  on. 
It  was  frustrating  not  to  be  able  to  change  the  model." 


COOKED  SAUSAGE 


One  member  felt  it  would  have  been  helpful  to  hold  at  least  one  meeting  of  the  committee.  "There  was 
often  a  person  missing  from  the  conference  call,  or  they  had  to  get  off  the  line  after  only  a  few  minutes 
to  take  care  of  other  business/' 

Another  said  "One  area  not  as  well  done  was  communication  within  the  committee.  I  have  not  seen  the 
final  copy  [of  the  generic  model],  but  only  a  fax/' 
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Steering  Committee  Role  and  Time,  Effort,  and  Costs  Expended 

Questionnaire  I:  Steering  Committee  Meetings  during  the  HACCP  Workshop 

This  questionnaire  focused  on  the  Steering  Committee's  activities  during  the  HACCP  Model 
Development  Workshop  in  Atlanta,  GA,  on  August  27-29, 1991.  A  Policy  Analysis  Unit  (PAU) 
Analyst  recorded  information  at  the  time  of  the  workshop  meeting.  A  PAU  Analyst  also  inter- 
viewed the  FSIS  Steering  Commiteee  Coordinator,  the  Steering  Committee  Chairperson,  and 
other  committee  members. 


Q-l.  How  many  times  did  the  Steering  Committee  meet? 

The  Poultry  Slaughter  Steering  Committee  met  briefly  once  at  the  end  of  the  Atlanta  workshop. 

Q-2.  Please  estimate  the  number  of  hours  spent  relative  to  each  of  these  meetings  concerning  actual  time  in 
meetings,  as  well  as  any  preparation  time  for  the  meetings. 

Time  in  Meeting         Preparation  Time 

Meeting  8/29/91  5-10  minutes  none 

Q-3.  In  general,  what  was  the  purpose  of  each  meeting?  What  topics  were  covered,?  What  was  accomplished  in 
each  meeting?  If  the(se)  meetings  resulted  in  any  decisions,  or  written  documentation,  report(s)  or  other 
products,  please  specify  and  estimate  the  number  of  hours  expended  in  these  activities  that  were  not 
included  in  the  response  to  Question  2  above. 

Dr.  Nathanial  Clark,  FSIS  Steering  Committee  Coordinator,  met  with  Steering  Committee  members 
for  about  5-10  minutes  at  the  conclusion  of  the  workshop.  He  explained  the  role  of  the  Steering 
Committee/  addressed  any  items  they  didn't  understand  and  oversaw  the  selection  of  the  chairperson. 
Mike  Gregory,  Hudson  Foods,  Inc.  volunteered  to  be  chairperson. 

In  addition  to  nwordsmithingn  the  model,  the  committee  was  asked  to  address  the  "notes  to  industry" 
on  chemical  residues  and  chlorination  which  were  left  unresolved  at  the  workshop. 


Q-4. 


Were  there  any  other  Steering  Committee  activities  that  occurred  during  the  workshop  that  haven't 
been  addressed?  If  yes,  please  explain. 


There  were  no  other  activities  during  the  workshop. 
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Steering  Committee  Role  and  Time,  Effort,  and  Costs  Expended 

Questionnaire  II:  Steering  Committee  Contacts  between  the  Workshop  and  the 

On-Site  Visit 

This  questionnaire  was  designed  to  collect  information  on  the  activities  of  the  Steering  Corni] 
tee  for  the  period  of  time  between  the  HACCP  Workshop  and  the  Pilot  Plant  On-Site  Evalual 
tion.  A  PAU  Analyst  administered  this  questionnaire  by  telephone  to  the  FSIS  Steering  Comij  tl 
tee  Coordinator,  the  Steering  Committee  Chairperson,  members  of  the  Steering  Committee.  |  h 

Q-l.  What  activities  were  undertaken  by  the  Steering  Committee  during  the  time  period  between  the  Worl 
and  the  On-Site  Evaluation  (e.g.,  meetings,  phone  calls,  etc.)?  What  was  the  purpose  of  each  activity? 
What  topics  were  covered  and  what  was  accomplished  by  each  activity?  How  much  time  was  spent  ira|.i,  W 
each  activity?  Was  there  preparation  time  involved?  If  so,  how  much  time? 

The  chairperson  talked  to  committee  members  individually  about  how  to  accomplish  their  task.  Oral 
suggested  that  each  committee  member  be  responsible  for  "wordsmithing"  a  column  in  the  generic  p ; 
such  as  "Critical  Limits"  or  "HACCP  System  Verification."  The  chairperson  polled  committee  memb  s 
about  mis  method  and  they  agreed  to  use  it  He  then  assigned  each  member  columns  from  the  geneil 
plan.  Committee  members  "wordsmithed"  their  assignments  and  mailed  or  faxed  the  drafts  to  him.  1 
The  chairperson  sent  them  a  final  draft  of  the  entire  plan  for  their  review  prior  to  submitting  it  to  the] 
Steering  Committee  Coordinator. 

As  his  assignment,  the  chairperson  addressed  the  "notes  from  industry."  According  to  him  workshop!^  v 
participants  had  argued  for  three  days  about  "chemical  residue"  and  "chlorination"  issues  and  finally!] 
assigned  the  Steering  Committee  the  task  of  deciding  what  to  do  about  these  items.  The  chairperson 
felt  that  since  the  workshop  participants  as  a  whole  had  added  these  two  concerns  as  notes  to  the 
generic  plan,  it  was  not  appropriate  for  the  Steering  Committee  to  add  them  to  the  model.  In  his 
submittal  letter,  the  committee  acknowledged  the  "notes  from  industry"  and  endorsed  them  as 
recommendations  to  USD  A. 

The  chemical  residue  note  is  an  acknowledgment  that  the  poultry  industry  already  has  in  place  effect |u  \ 
preventative  and  monitoring  programs  in  the  live  production  area.  The  inclusion  of  chemical  residue] 
as  a  CCP  at  slaughter  would  be  a  duplication  of  effort  and  less  effective. 

The  chlorination  note  is  a  series  of  recommended  check  points  to  be  considered  as  the  CCP's  regardin 
chlorination  of  the  chillers  are  implemented  during  the  pilot  plant  studies. 

The  committee  submitted  the  polished  model  to  the  FSIS  Steering  Committee  Coordinator  on 
November  15, 1991. 

Q-2.  Was  the  entire  Steering  Committee  involved  in  these  activities  (meetings,  telephone  calls,  report  writing, 
etc.)?  Please  elaborate.  Yes  X  No 

All  committee  members  performed  their  assignments. 

Q-3.  Was  anyone  other  than  Steering  Committee  members  involved  in  these  activities? 

Yes  X       No  (Continue  with  Q4| 

If  yes,  please  elaborate. 

The  chairperson  and  one  other  member  talked  to  the  chairperson  of  the  Cooked  Sausage  Steering 
Committee.  Stacey  Edelbach,  Training  Staff,  typed  the  polished  model  for  the  committee  prior  to 
submission  to  the  Steering  Committee  Coordinator. 

Q-4.  Were  there  any  other  Steering  Committee  activities  that  occurred  during  this  time  period  that  we  haven't 
discussed?  Yes   No  X 


If  yes,  please  explain. 


POULTRY  SLAUGHTER 
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?  Steering  Committee  Activities 


^  2-1.  What  activities  were  undertaken  by  the  Steering  Committee  during  and  after  the  On-Site  Evaluation 

(e.g.,  meetings,  phone  calls,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics  were  covered  and 
what  was  accomplished  by  each  activity?  How  much  time  was  spent  in  each  activity?  Was  there 
preparation  time  involved?  If  so,  how  much  time?  [N.B.:  If  no  activities  occurred,  skip  to  Q-4.] 
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Steering  Committee  Role  and  Time,  Effort,  and  Costs  Expended 

Questionnaire  III:  Steering  Committee  Contacts  during  and  after  the  On-Site  Visit 

This  questionnaire  requested  information  on  the  Steering  Committee's  activities  during  and 
after  the  pilot  plant  On-Site  Evaluation.  It  also  addressed  the  role  of  the  Steering  Committee  in 
general.  This  instrument  was  adminstered  by  the  Policy  Analysis  Unit  by  telephone  to  the  FSIS 
Steering  Committee  Coordinator  and  to  all  Steering  Committee  members  upon  completion  of 
the  On-Site  Evaluation  or  after  all  (if  any)  refinements /changes  to  the  Generic  HACCP  Model 
have  been  incorporated. 


The  committee  had  no  activities  during  the  On-Site  Evaluation  period. 


ksln        Was  the  entire  Steering  Committee  involved  in  these  activities  (meetings,  telephone  calls,  report  writing, 


etc)?  Please  elaborate.  Yes   No. 

N/A 


^  J-3.  Was  anyone  other  than  Steering  Committee  members  involved  in  these  activities? 


Yes   No  (Continue  with  Q-4) 

If  yes,  please  elaborate. 


N/A 


2-4.  Were  there  any  other  Steering  Committee  activities  that  occurred  during  this  time  period  that  we  haven't 
discussed?  Yes   No  


i 


If  yes,  please  explain. 
N/A 

Steering  Committee  Role 

Q-5.  Now  that  you  have  served  as  a  member  of  the  Steering  Committee,  what  do  you  think  is  the  role  of  the 
Steering  Committee? 

I Committee  members  felt  the  role  of  the  committee  was  to  "wordsmith"  the  generic  model.  The 
chairperson  said  some  steering  committees  tried  to  come  up  with  separate  models.  He  said  "as 
representatives  of  the  workshop  participants,  we  [the  Poultry  Slaughter  Steering  Committee]  should 
respect  the  actions  of  the  workshop  participants  and  not  recommend  changes." 


POULTRY  SLAUGHTER 


One  complained  that  basically  the  committee  was  doing  secretarial  work.  He  felt  that  if  the  committt 
was  elected  by  the  workshop  participants,  then  the  members  should  be  able  to  make  changes  in  the 
model. 


Q-6.  In  brief,  what  did  you  do  to  perform  your  role? 

The  chairperson  talked  one-on-one  with  committee  members  and  assigned  columns  to  "wordsmith," 
Assignments  were  given  based  on  areas  that  members  had  specialized  in  at  the  workshop. 

Committee  members  "wordsmithed"  their  columns  and  faxed  them  to  the  chairperson.  One  member  i 
said  he  used  reference  information  for  his  work  on  'Verification"  and  had  read  the  other  HACCP 
generic  models. 


Q-7.  Please  note  any  other  comments  you  would  like  to  offer  concerning  the  role,  time  and  effort  of  the 
Steering  Committee.  Consider  usefulness  of  the  Committee  in  polishing  and/or  refining  the  Generic 
HACCP  Model,  any  benefits  to  you  personally  and /or  your  company,  travel  expenses,  costs  of 
telephone  calls,  etc. 

The  chairperson  estimated  that  he  had  spent  approximately  16  hours  on  Steering  Committee  business,  I 
His  company  was  supportive  of  his  work  with  HACCP.  He  said  his  biggest  problem  was  contacting 
committee  members.  In  hindsight,  he  felt  he  probably  should  have  set  a  date  for  a  conference 
call  or  meeting  to  hammer  it  out  ["wordsmithing  the  generic  model"!.  It  might  have  been  done  more 
efficiently  that  way  rather  than  his  trying  to  catch  up  with  people  by  phone.  He  said  his  team  were  all 
so  busy  traveling  that  he  had  a  hard  time  keeping  up  with  them. 

He  felt  more  direction  could  have  been  given  about  how  the  committee  should  function.  'It  would  be 
helpful  to  have  some  tips  to  find  the  best  way."  He  would  like  to  be  given  ideas  about  how  the  group 
should  function  if  he  should  have  to  reconvene  his  team  after  the  on-site  visits. 


Committee  members  spent  from  4  to  12  hours  on  their  asignments.  Their  companies  were  supportive 
of  their  involvement  One  said  his  company  was  pushing  for  it  [HACCP]  and  had  done  a  lot  of  work 
and  training  in  this  area.  None  felt  their  involvement  was  a  burden  on  their  company  but  was  rather 
a  plus  to  the  company. 

One  member  had  been  on  the  National  Broiler  Council  Subcommittee  on  HACCP  so  he  was  interested 
in  going  to  the  workshop.  '1  am  a  strong  supporter  of  the  program.  Ifs  the  first  time  I'd  been  on  a 
committee.  It  was  easier  that  trying  to  persuade  one  of  our  plants  to  be  a  pilot  project" 

One  member  said  he  thought  the  generic  plan  was  very  workable.  He  said  his  company  has  installed 
the  generic  model  in  their  plants. 

Q-8,  What  are  your  ideas  concerning  the  usefulness  and  viability  of  the  function  of  the  Steering  Committee? 

One  committee  member  said  it  was  important  for  industry  to  participate.  "They  [industry  members] 
know  what  will  work  in  a  plant  They  [industry  members]  can  do  a  better  selling  job  on  HACCP." 

One  was  disturbed  by  the  fact  that  FSIS  representatives  voted  at  the  HACCP  workshop.  He  suggested 
that  an  alternative  way  of  developing  a  HACCP  model  would  be  for  an  advisory  panel  to  put  one 
together.  He  felt  the  workshop  was  biased  by  interest  groups  like  the  Turkey  Council,  The  National 
Broiler  Council,  and  US  DA, 

Although  his  company  is  strongly  in  favor  of  HACCP,  this  same  member  didn't  think  HACCP  would 
be  used  by  the  industry  without  USD  A  pushing  for  it 
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Steering  Committee  Role  and  Time,  Effort,  and  Costs  Expended 

Questionnaire  I:  Steering  Committee  Meetings  during  the  HACCP  Workshop 

This  questionnaire  focussed  on  the  Steering  Committee's  activities  during  the  HACCP  Model 
Development  Workshop  at  Phoenix,  AZ,  on  December  3-5, 1991.  A  PAU  Analyst  interviewed 
by  telephone  the  FSIS  Steering  Commiteee  Coordinator,  the  Steering  Committee  Chairperson, 
and  other  committee  members. 


Q-L  How  many  times  did  the  Steering  Committee  meet? 

The  Ground  Beef  Steering  Committee  met  briefly  at  the  Phoenix  workshop  before  the  final  lunch. 

Q-2.  Please  estimate  the  number  of  hours  spent  relative  to  each  of  these  meetings  concerning  actual  time  in 
meetings,  as  well  as  any  preparation  time  for  the  meetings. 

Time  in  Meeting         Preparation  Time 

Meeting  12/5/91  5-10  minutes  none 

Q-3.  In  general,  what  was  the  purpose  of  each  meeting?  What  topics  were  covered,?  What  was  accomplished  in 
each  meeting?  If  the(se)  meetings  resulted  in  any  decisions,  or  written  documentation,  report(s)  or  other 
products,  please  specify  and  estimate  the  number  of  hours  expended  in  these  activities  that  were  not 
included  in  the  response  to  Question  2  above. 


Dr.  Nathaniel  Clark,  FSIS  Steering  Committee  Coordinator,  met  with  the  committee.  He  explained 
the  role  of  the  Steering  Committee,  answered  their  questions,  and  oversaw  the  selection  of  chairperson, 
Ryan  Englund,  Jac  Pac  Foods,  and  co-chairperson,  William  Lloyd,  Monfort,  Inc.  The  chairperson 
assigned  duties  to  the  committee  and  a  date  was  set  for  a  conference  call. 

The  committee  was  not  charged  with  any  tasks  other  than  "word smithing"  the  model  developed  at  the 
workshop. 

(H  Were  there  any  other  Steering  Committee  activities  that  occurred  during  the  workshop  that  haven't 
been  addressed?  If  yes,  please  explain. 

There  were  no  other  activities  during  the  workshop. 


GROUND  BEEF 


Steering  Committee  Role  and  Time,  Effort,  and  Costs  Expended 

Questionnaire  II:  Steering  Committee  Contacts  between  the  Workshop  and  the 

On-Site  Visit 

This  questionnaire  was  designed  to  collect  information  on  the  activities  of  the  Steering  Comm 
tee  for  the  period  of  time  between  the  HACCP  Workshop  and  the  Pilot  Plant  On-Site  Evalua- 
tion. A  PAU  Analyst  administered  this  questionnaire  by  telephone  to  the  FSIS  Steering  Comr. 
tee  Coordinator,  the  Steering  Committee  Chairperson,  and  other  members  of  the  Steering  Cor 
mittee. 


Q-l.  What  activities  were  undertaken  by  the  Steering  Committee  during  the  time  period  between  the  Works 
and  the  On-Site  Evaluation  (e.g.,  meetings,  phone  calls,  etc.)?  What  was  the  purpose  of  each  activity? 
What  topics  were  covered  and  what  was  accomplished  by  each  activity?  How  much  time  was  spent  in 
each  activity?  Was  there  preparation  time  involved?  If  so,  how  much  time? 

Each  member  of  the  committee  had  been  asked  to  make  notations  and  changes  to  the  generic  model 
and  send  them  to  the  chairperson  by  the  Tuesday,  December  10, 1991,  following  the  workshop.  He 
combined  their  suggestions  into  a  rough  draft  and  sent  it  out  to  the  committee  members.  At  a  tele- 
conference on  December  19, 1991  (of  about  one-half  hour),  members  went  through  the  model 
word-by-word  and  made  further  changes.  The  chairperson  typed  in  their  changes  and  sent  committe 
members  the  final  draft  for  review.  He  submitted  the  final  version,  dated  December  19, 1991,  to  the 
FSIS  Steering  Committee  Coordinator. 

The  committee  felt  under  pressure  as  they  had  been  urged  by  the  FSIS  Steering  Committee  Coordina 
to  complete  their  "wordsmithing"  as  quickly  as  possible. 

Q-2.  Was  the  entire  Steering  Committee  involved  in  these  activities  (meetings,  telephone  calls,  report  writing,; 
etc.)?  Please  elaborate.  Yes  X_  No  


All  committee  members  made  changes  or  notes  on  the  draft  models  they  sent  to  the  chairperson. 
The  members  tried  as  much  as  possible  to  reflect  the  desires  of  the  workshop  participants.  They 
wanted  to  produce  consistency,  clarity,  and  brevity. 


Q-3.  Was  anyone  other  than  Steering  Committee  members  involved  in  these  activities? 

Yes  __X_  No__(Continue  with  Q-4) 

If  yes,  please  elaborate. 

One  member  shared  the  generic  plan  with  the  Quality  Control  staff  at  his  plant  as  they  are  working 
on  their  own  HACCP  plan. 


Q-4.  Were  there  any  other  Steering  Committee  activities  that  occurred  during  this  time  period  that  we  haven't 
discussed?  Yes  X   No  

If  yes,  please  explain. 

The  chairperson  said  he  had  gotten  a  diskette  with  a  copy  of  the  model  on  it  which  made  the  task 
of  updating  the  drafts  go  relatively  smoothly.  He  also  said  he  had  shared  the  model  with  people 
in  his  plant  but  had  not  asked  them  for  help  with  the  "wordsmithing." 
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Steering  Committee  Role  and  Time,  Effort,  and  Costs  Expended 

Questionnaire  III:  Steering  Committee  Contacts  during  and  after  the  On-Site  Visit 

This  questionnaire  requested  information  on  the  Steering  Committee's  activities  during  and 
after  the  pilot  plant  On-Site  Evaluation.  It  also  addressed  the  role  of  the  Steering  Committee  in 
general.  This  instrument  was  adminstered  by  the  Policy  Analysis  Unit  by  telephone  to  the  FSIS 
Steering  Committee  Coordinator  and  to  all  Steering  Committee  members  upon  completion  of 
the  On-Site  Evaluation  or  after  all  (if  any)  refinements/changes  to  the  Generic  HACCP  Model 
have  been  incorporated. 


Q-l.  What  activities  were  undertaken  by  the  Steering  Committee  during  and  after  the  On-Site  Evaluation 

(e.g.,  meetings,  phone  calls,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics  were  covered  and 
what  was  accomplished  by  each  activity?  How  much  time  was  spent  in  each  activity?  Was  there 
preparation  time  involved?  If  so,  how  much  time?  [N.B.:  If  no  activities  occurred,  skip  to  Q-4J 


There  were  no  peer  reviewed  pilot  tests  of  the  Ground  Beef  generic  model. 


iidinjQ"2*  Was  the  entire  Steering  Committee  involved  in  these  activities  (meetings,  telephone  calls,  report  writing, 
etc.)?  Please  elaborate.  Yes   No  


N/A 

Q-3.  Was  anyone  other  than  Steering  Committee  members  involved  in  these  activities? 

Yes   No  (Continue  with  Q-4) 

N/A 


Q-4.  Were  there  any  other  Steering  Committee  activities  that  occurred  during  this  time  period  that  we  haven't 
discussed?  Yes   No  


N/A 


Steering  Committee  Role 


Q-5.  Now  that  you  have  served  as  a  member  of  the  Steering  Committee,  what  do  you  think  is  the  role  of  the 
Steering  Committee? 

Committee  members  thought  the  role  of  the  steering  committee  was  to  refine  the  model  for  consistency. 

The  chairperson  thought  the  group  should  meet  after  the  one-site  evaluation  period  to  deal  with 
portion  control  for  steaks. 


GROUND  BEEF 


"We  worked  easily  compared  to  some  groups.  One  reason  may  have  been  because  of  the  type  of 
product  Ground  beef  doesn't  have  too  many  steps  compared  to  some  other  processes." 

'If  s  [the  Steering  Committee]  very  important  Industry  should  regulate  itself." 
Q-6.  In  brief,  what  did  you  do  to  perform  your  role? 

"I  made  comments  on  the  draft  model  and  faxed  them  to  the  chairperson  and  participated  in  the 
conference  call." 

"As  chairperson,  I  edited  together  all  the  committee's  comments  and  sent  out  for  review  the  final  draft 
before  submitting  it  to  the  Steering  Committee  Coordinator.  I  organized  the  conference  call." 

Q-7.  Please  note  any  other  comments  you  would  like  to  offer  concerning  the  role,  time  and  effort  of  the 
Steering  Committee.  Consider  usefulness  of  the  Committee  in  polishing  and/or  refining  the  Generic 
HACCP  Model,  any  benefits  to  you  personally  and/or  your  company,  travel  expenses,  costs  of 
telephone  calls,  etc. 

Committee  members  estimated  that  they  had  spent  from  2-8  hours  doing  work  for  the  Steering 
Committee.  The  cost  was  minimal  to  their  companies.  Their  companies  were  enthusiastic  about  their 
participation.  One  member  mentioned  that  his  company  was  thrilled  because  they  wanted  to  be  a  pilot 
plant 

The  chairperson  said  he  had  spent  about  three  days  total  on  Steering  Committee  business.  If  he  had 
had  to  attend  a  meeting,  his  participation  on  the  committee  would  have  been  a  burden  to  his  company, 
He  was  absolutely  sold  on  the  program  as  his  company  is  doing  it  anyway. 

He  said  the  Ground  Beef  Steering  Committee  had  an  advantage  over  the  other  groups.  They  were  able 
to  follow  the  standards  set  by  earlier  groups  so  it  ["wordsmithing"  the  model]  went  very  smoothly. 

The  chairperson's  company  is  very  involved  in  quality.  Ten  percent  of  bonuses  are  based  on  quality. 
They  must  make  their  product  right  the  first  time  or  costs  become  a  problem.  They  are  moving  into 
HACCP  and  would  like  to  be  a  pilot  plant 

"It  didn't  matter  for  the  working  of  the  committee  what  size  firm  members  came  from.  It  may  be  more 
difficult  for  smaller  firms  to  implement  HACCP,  however." 

Q-8.  What  are  your  ideas  concerning  the  usefulness  and  viability  of  the  function  of  the  Steering  Committee? 

"The  Steering  Committee  is  necessary.  It  would  not  work  to  have  a  USDA  group  do  it  [refine  the  generic 
model].  If  industry  does  it  [refine  the  model],  it  gives  them  more  ownership  of  the  model." 

One  member  would  like  to  see  the  Steering  Committee  visit  a  pilot  plant  to  see  how  the  model  was 
working  although  he  recognised  the  impossibility  of  doing  so  since  each  plant  would  want  to  protect  its 
proprietary  information. 

The  chairperson  wished  more  people  from  industry  had  attended  the  workshop.  He  said  the  approach 
was  good.  It  would  be  hard  for  industry  to  do  it  [develop  a  generic  model]  by  itself.  Most  industry 
representatives  don't  have  the  scientific  background.  It  was  a  cooperative  effort  which  will  be 
beneficial  for  the  future. 

One  member's  company  had  considered  being  a  pilot  plant  but  was  leery  about  volunteering  because 
the  company  is  participating  in  an  organic  acid  wash  pilot  which  is  now  in  its  8th  year  of  operation. 

One  suggested  several  options  other  than  the  workshop  to  develop  HACCP  plans.  "One  method  would 
be  for  FSIS  to  put  HACCP  models  for  comment  in  the  Federal  Register.  At  the  workshops,  FSIS 
already  knew  what  it  wanted."  Another  idea  he  had  was  to  take  a  process  that  many  PQC  programs  have 
been  written  for  and  write  a  HACCP  model  based  on  them.  He  also  suggested  setting  up  a  steering 
committee  for  each  product 
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Steering  Committee  Roles,  and  Time,  Effort,  and  Costs  Expended 

Questionnaire  I:  Steering  Committee  Meetings  during  the  HACCP  Workshop 

This  questionnaire  focuses  on  the  Steering  Committee's  activities  during  the  HACCP  Model 
Development  Workshop  at  Minneapolis,  MN  on  March  31  -  April  2, 1992.  A  Policy  Analysis 
Unit  (PAU)  analyst  recorded  information  at  the  time  of  the  workshop  meeting.  A  PAU  analyst 
also  interviewed  the  FSIS  Steering  Commitee  Coordinator,  Steering  Committee  Chairperson, 
and  other  committee  members. 


Q-l.  How  many  times  did  the  Steering  Committee  meet? 

The  Swine  Slaughter  Steering  Committee  met  once  briefly  at  the  end  of  the  Minneapolis  workshop. 

Q-2.  Please  estimate  the  number  of  hours  spent  relative  to  each  of  these  meetings  concerning  actual  time  in 
meetings,  as  well  as  any  preparation  time  for  the  meetings. 

Time  in  Meetings         Preparation  Time 

Meeting  4/2/92  15-20  minutes  none 

Q-3.  In  general,  what  was  the  purpose  of  each  meeting?  What  topics  were  covered?  What  was  accomplished  in 
each  meeting?  If  the(se)  meetings  resulted  in  any  decisions,  or  written  documentation,  report(s)  or  other 
products,  please  specify  and  estimate  the  number  of  hours  expended  in  these  activities  that  were  not 
included  in  the  response  to  Question  2  above. 

Steering  committee  members  introduced  themselves.  Dr.  Nathaniel  Clark,  FSIS  Steering  Committee 
Coordinator,  explained  the  duties  of  the  committee  including  what  could  and  could  not  be  changed  in 
the  model.  A  chairperson  and  co-chairperson  were  named.  The  committee  asked  Katie  Hanigen, 
Farmland  Foods,  to  be  chairperson.  Ellen  Gxaber,  Thorn  Apple  Valley,  was  named  co-chairperson. 
The  chairperson  told  the  others  she  would  be  in  contact  with  themto  set  up  a  conference  call  In 
the  meantime,  committee  members  were  to  study  their  hard  copies  of  the  model  and  make  any  changes 
in  preparation  for  the  calL  The  members  were  "scrambling  to  catch  airplanes"  so  the  meeting  was  short 

The  workshop  participants  wanted  the  Steering  Committee  to  address  two  issues  relative  to  food  safety 
as  addenda  to  the  generic  plan: 

pesticides  and  antibiotic  residues  and 

organic  acid  sanitizing  systems  for  microbial  reduction. 


Q-4.  Were  there  any  other  Steering  Committee  activities  that  occurred  during  the  workshop  that  haven't 
been  addressed?  If  yes,  please  explain. 

There  were  no  other  activities  during  the  workshop. 
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Questionnaire  II:  Steering  Committee  Contacts  between  the  Workshop  and  the 

On-Site  Visit 

This  questionnaire  was  designed  to  collect  information  on  the  activities  of  the  Steering  Comn 
tee  for  the  period  of  time  between  the  HACCP  Workshop  and  the  Pilot  Plant  On-Site  Visit.  A 
PAU  Analyst  administered  this  questionnaire  by  telephone  to  the  FSIS  Steering  Committee 
Coordinator,  the  Steering  Committee  Chairperson,  and  members  of  the  Steering  Committeee  i 

Q-l.  What  activities  were  undertaken  by  the  Steering  Committee  during  the  time  period  between  the 

Workshop  and  the  On-Site  Visit  {e.g.,  meetings,  phone  calls,  etc.)?  What  topics  were  covered  and  what 
was  accomplished  by  each  activity?  How  much  time  was  spent  in  each  activity?  Was  there  preparatioi 
time  involved?  If  so,  how  much  time?  Overall,  how  significant  was  each  issue  that  was  addressed. 

The  chairperson  contacted  Dr.  Clark  to  verify  the  charge  to  the  committee.  She  then  set  up  a  confere 
call  which  was  held  April  22, 1992,  for  two  hrs.  and  15  min.  At  the  conference  call,  the  committee  w< 
through  each  step  of  the  model  and  discussed  each  point  and  committee  members  agreed,  disagreed, 
made  suggestions,  and  then  voted.  After  this  conference  call,  the  chairperson  sent  out  a  draft  revisio 
and  the  committee  held  another  conference  call  on  May  6, 1992,  for  15  minutes  to  go  over  the  revised 
document  In  addition,  other  phone  calls  were  made  between  members.  Only  a  few  minor  changes 
were  made  after  the  May  6  conference  call. 

The  chairperson  then  sent  the  final  version  to  Dr.  Clark  on  May  7, 1992,  which  included  the  addenda 
written  by  Kirk  Jones,  one  of  the  committee  members.  The  addendum  on  pesticides  and  antibiotic 
residues  suggested  that  cooperation  among  regulatory  agencies,  swine  producers,  and  packers  is 
necesssary  to  reduce  the  chemical  residue  hazard.  It  listed  present  and  future  preventative  methods 
which  could  be  taken.  The  addendum  on  organic  acid  sanitizing  systems  agreed  that  add  sprays  are 
not  a  substitute  for  Good  Manufacturing  Practices,  but  would  be  useful  as  a  means  of  intervention  an 
prevention  of  bacterial  growth  during  and  after  the  slaughter  process.  It  then  listed  points  in  favor  of 
using  organic  add  sprays. 

Dr.  Clark  sent  a  letter  to  the  committee  acknowledging  receipt  of  the  polished  model  and  thanking 
committee  for  its  efforts. 

Q-2.  Was  the  entire  Steering  Committee  involved  in  these  activities  (meetings,  telephone  calls,  report  writing,] 
etc.)?  Please  elaborate.  Yes  X  No. 

All  members  participated  in  both  conference  calls. 


Sleeri 
Q-l. 


& 


Q-3. 


Q-3.  Was  anyone  other  than  Steering  Committee  members  involved  in  these  activities? 

Yes    X      No  (Continue  with  Q-| 

If  yes,  please  elaborate. 

Some  members  talked  about  the  model  with  other  employees  at  their  plants.  The  chairperson's  secret 
typed  drafts  of  the  model. 

Q-4.  Were  there  any  other  Steering  Committee  activities  that  occurred  during  this  time  period  that  we  haven't 
discussed?  Yes  X  No  

If  yes,  please  explain. 

On  June  12, 1992,  the  chair  received  a  call  from  Dr.  Clark.  Training  staff  at  Texas  A&M  had  contacted 
him  asking  that  footnotes  be  added  to  the  generic  model.  Operational  Sanitation  was  described  as  CC 
9B  near  the  end  of  the  model,  but  was  used  on  earlier  pages  of  the  model.  The  requested  footnote 
"*  Covered  on  pg.  11  under  96"  was  to  be  added  wherever  Operational  Sanitation  was  mentioned  undt  j 
Preventative  Measures.  The  chairperson  made  the  appropriate  changes,  returned  a  copy  to  Dr.  Clark, 
and  contacted  her  committee  about  the  changes,  ready  to  initate  another  conference  call  if  the  committi 
felt  it  was  necessary. 
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Questionnaire  III:  Steering  Committee  Contacts  during  and  after  the  On-Site  Visit 

This  questionnaire  requested  information  on  the  Steering  Committee's  activities  during  and 
after  the  pilot  plant  On-Site  Visit.  It  also  addressed  the  role  of  the  Steering  Committee  in 
general.  This  instrument  was  adminstered  by  the  Policy  Analysis  Unit  by  telephone  to  the  FSIS 
Steering  Committee  Coordinator,  Assistant  Coordinator,  and  all  Steering  Committee  members 
upon  completion  of  the  On-Site  Evaluation  or  after  all  (if  any)  refinements /changes  to  the 
Generic  HACCP  Model  have  been  incorporated. 

Steering  Committee  Activities 

Q-l.  What  activities  were  undertaken  by  the  Steering  Committee  during  and  after  the  On-Site  Visit 

(e.g.,  meetings,  phone  calls,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics  were  covered  and 
what  was  accomplished  by  each  activity?  How  much  time  was  spent  in  each  activity?  Was  there 
preparation  time  involved?  If  so,  how  much  time?  [N.B.:  If  no  activities  occurred,  skip  to  Q-4.] 

There  were  no  peer  reviewed  pilot  tests  of  the  Swine  Slaughter  generic  model. 


Q-2.  Was  the  entire  Steering  Committee  involved  in  these  activities  (meetings,  telephone  calls,  report  writing, 
etc.)?  Please  elaborate.  Yes   No  


N/A 


Q-3.  Was  anyone  other  than  Steering  Committee  members  involved  in  these  activities? 

Yes   No  (Continue  with  Q-4) 

If  yes,  please  elaborate. 
N/A 


Q-4.  Were  there  any  other  Steering  Committee  activities  that  occurred  during  this  time  period  that  we  haven't 
discussed?  Yes   No  

If  yes,  please  explain. 

The  chair  had  served  on  the  HACCP  Cooked  Sausage  Steering  Committee  also. 
Steering  Committee  Role 

Q-5.  Now  that  you  have  served  as  a  member  of  the  Steering  Committee,  what  do  you  think  is  the  role  of  the 
|        Steering  Committee? 

"The  role  of  the  Steering  Committee  is  to  put  consistent  wording  throughout  the  model  and  eliminate 
anything  that  would  tie  a  company  down  to  using  a  specific  method  to  control  at  a  CCP." 

"Someone  had  to  "wordsmith"  the  model  and  make  the  wording  more  uniform.  People  at  the  workshop 
who  had  been  at  other  workshops  referred  to  the  other  models  for  wording,  but  this  Steering  Committee 
did  not  refer  to  other  models  during  our  discissions," 
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'The  role  of  the  Steering  Committee  was  very  important  It  would  be  difficult  to  get  a  large  group 
like  the  entire  workshop  to  agree.  A  small  group  could  get  changes  in  wording  done  quicker  and 
easier." 

"It  was  easier  for  me  to  work  with  a  group  than  perhaps  others  on  the  committee  as  my  company 
works  in  teams  all  the  time." 

Q-6.  In  brief,  what  did  you  do  to  perform  your  role? 

"I  set  up  conference  calls,  faxed  information  about  the  calls  and  generic  model  to  committee  members, 
and  contacted  Dr.  Clark  with  questions.  I  tried  to  avoid  having  a  meeting  as  the  members  are  busy 
people." 

Some  additional  faxes  were  sent  between  committee  members  with  information  about  the  model. 
Stacey  Edelbach  at  Texas  A&M  sent  a  disk  with  the  model  on  it  so  the  chairperson's  secretary  could 
make  the  modifications  to  it 

Committee  members  prepared  ahead  for  conference  calls,  participated  in  the  conference  calls,  and  madi 
modifications  to  the  model. 

"  I  prepared  the  addenda  on  chemical  residues  from  pesticides  and  antibiotics  and  on  organic  acid 
sanitizing  systems." 

Q-7.  Please  note  any  other  comments  you  would  like  to  offer  concerning  the  role,  time  and  effort  of  the 
Steering  Committee.  Consider  usefulness  of  the  Committee  in  polishing  and/or  refining  the  Generic 
HACCP  Model,  any  benefits  to  you  personally  and/or  your  company,  travel  expenses,  costs  of 
telephone  calls,  etc. 

"I  spent  2  or  3  days  on  Steering  Committee  business.  My  secretary  spent  about  8  hours  faxing  memos 
to  committee  members  and  typing  the  changes  to  the  model  that  Stacey  had  sent  up  on  disk.  Cost  of  the 
two  conference  calls  was  $275.  My  company  has  been  supportive  and  is  really  behind  my 
participation  in  HACCP." 

Committee  members  estimated  they  had  spent  between  3  and  20  hours  on  Steering  Committee 
business.  None  felt  their  participation  on  the  Steering  Committee  was  a  burden  to  their  companies. 
Rather,  their  companies  were  supportive  of  their  participation. 


Q-8.  What  are  your  ideas  concerning  the  usefulness  and  viability  of  the  function  of  the  Steering  Cornmittee? 

"It  is  important  to  have  a  steering  committee  of  industry  members.  If  the  government  did  the  "word* 
smithing",  some  people  would  complain.  Workshop  participants  have  a  more  comfortable  feeling 
leaving  the  meeting  knowing  that  IBP,  Montf  ort,  Farmland,  Thorn  Apple  Valley,  and  Johnsonville 
Foods  would  be  doing  the  "worksmithing"." 

Representatives  of  both  large  and  small  companies  attended  the  workshop  so  the  interests  of  the  smaller) 
packers  were  taken  into  account  There  was  a  good  variety  of  experiences  on  the  Steering  Committee. 
Some  were  nitty,  gritty  operations  types  at  the  plant  manager,  supervisor  level  who  were  knowledgable 
and  could  provide  a  reality  check  on  the  steps  in  the  model. 

"Our  company  will  begin  implementing  our  own  HACCP  program  on  the  kill  floor.  We  will  try  it 
for  12-16  weeks  to  see  if  it  makes  a  difference.  I  asked  my  company  to  try  HACCP  as  there  are  not 
going  to  be  any  pilot  plants.  Our  plant  will  have  some  valuable  data  lo  use  for  making  comments  if 
regulations  come  out  on  HACCP.  I  talked  to  Dr.  Clark  about  it  He  said  we  should  use  all  the  generic 
CCPs  and  any  others  which  apply  directly  to  our  plant" 

"I  think  that  HACCP  applies  better  and  is  more  effective  on  a  processed  product  like  cooked  sausage 
than  on  the  kill  floor." 
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"The  Steering  Committee  worked  well  together.  There  was  a  good  cross  section  of  the  pork  business: 
2  large,  2  mid-size  and  one  small  company  were  represented.  We  developed  good  relationships  among 
committee  members.  I'm  relatively  new  in  the  industry  so  it  helped  to  get  to  know  and  work  on  a 
common  interest  with  others  in  the  industry." 

"It  was  important  to  develop  a  generic  model  which  provides  some  consistancy  for  the  industry.  Since 
representatives  of  industry  had  a  hand  in  developing  it,  it  is  apt  to  be  more  palatable  to  industry. 
Getting  consensus  is  very  important  The  workshop  process  worked  well." 

'1  would  like  to  see  the  Steering  Committee  become  a  standing  committee.  It  could  be  a  forum  for 
dealing  with  industry-wide  issues  which  may  arise  in  the  slaughter  of  market  hogs  such  as  branding 
of  carcasses  and  neck  wounds." 

"Our  company  has  HACCP  plans  in  effect  already  so  were  had  a  jump  on  the  others  as  far  as  under- 
standing and  implementing  HACCP." 

"I  was  initially  hesitant  about  attending  the  workshop.  However,  it  worked  well  and  was  a  great 
forum." 
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HACCP 
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Overview  of 
Plant  Adaption  Activities 


In  January  1990,  the  Food  Safety  and  Inspection  Service  (FSIS)  of 
the  U.S.  Department  of  Agriculture  initiated  efforts  to  determine 
how  to  implement  the  Hazard  Analysis  and  Critical  Control  Point 
(HACCP)  system  in  regulated  meat  and  poultry  plants.  HACCP 
is  a  process  control  system  that  identifies  " critical H  points  in  the 
food  production  process  that  should  be  controlled  to  prevent  food 
safety  hazards  (i.e.,  microbiological,  chemical,  and  physical 
hazards)  from  occurring. 

Agency  activities  included  consultations  and  public  hearings  to 
explain  HACCP  and  the  Agency's  efforts,  workshops  to  develop 
model  HACCP  plans,  in-plant  testing  to  trial  the  specific  plant 
HACCP  plans  at  nine  volunteer  plants,  and  an  assessment  of  the 
overall  experience  of  the  Agency. 

A  series  of  reports  will  be  written  on  the  various  phases  and 
activities  undertaken  by  the  Agency  and  participants  during  the 
study.  This  is  one  of  the  reports.  For  further  information 
concerning  FSIS  HACCP  activities  contact: 

Dr.  Dorothy  Stringfellow 
HACCP  Office 

14th  &  Independence  Ave. ,  SW 
Room  4444 

Washington,  DC  20250 
Phone:  (202)  690-2087 
FAX:  (202)  690-3886 


HACCP  WORKSHOPS  REPORT 


Overview  of 
PLANT  ADAPTION  ACTIVITIES 


Vol  IV 


United  States  Department  of  Agriculture 
Food  Safety  and  Inspection  Service 
Policy  Analysis  Unit 
Washington,  D.C.  20250 
February  1994 


SUMMARY 


In  January  1990,  the  Food  Safety  and  Inspection  Service  (FSIS),  USDA,  began  a  study  to  determine  the 
process  for  HACCP*  implementation  into  meat  and  poultry  inspection  operations.  Five  workshops  were 
held  to  develop  generic  HACCP  plans  for  Refrigerated  Foods,  Cooked  Sausage,  Poultry  Slaughter,  Ground 
Beef,  and  Swine  Slaughter.  Following  the  Refrigerated  Foods,  Cooked  Sausage,  and  Poultry  Slaughter 
workshops,  the  generic  models  developed  at  these  workshops  were  tested  at  three  volunteer  plants  for  each 
product/process.  An  on-site  visit  was  conducted  by  two  members  of  the  HACCP  Special  Team  at  each 
pilot  plant  to  help  plant  management  develop  a  plant-specific  HACCP  plan. 

As  originally  designed,  a  volunteer  plant's  plant-specific  plan  was  to  be  compared  with  the  workshop- 
developed  generic  model  during  an  on-site  visit  at  the  pilot  plant  by  the  HACCP  Special  Team.  However, 
this  process  evolved  into  a  lengthier  procedure  as  it  became  obvious  that  plants  were  not  prepared  to  share 
a  plant-specific  plan  at  the  initial  on-site  meeting.  Instead,  an  on-site  orientation  at  the  plant  led  by 
Special  Team  members  initiated  the  process  of  developing  an  acceptable  plant-specific  HACCP  plan.  After 
the  orientation  session,  baseline  data  could  be  collected  at  the  plant  while  plant  management  completed 
the  HACCP  plan  over  a  period  of  three  or  more  months. 

The  generic  plans  developed  at  the  workshops  were  found  to  be  applicable  to  the  plants.  The  Special 
Team  requested  only  one -change  to  a  generic  plan  based  on  plant-specific  plans  in  the  pilot  plants.  All 
the  Refrigerated  Foods  plant-specific  plans  contained  a  critical  control  point  (CCP)  for  "receiving"  which 
was  not  included  in  the  generic  plan  for  Refrigerated  Foods.  The  Special  Team  requested  the  Refrigerated 
Foods  Workshop  Steering  Committee  act  to  include  a  CCP  for  "receiving"  in  the  generic  model. 

HACCP  Special  Team  members  spent  a  minimum  of  five  days  at  each  plant  for  the  on-site  orientation  and 
later  visits  to  help  plant  personnel  develop  their  plant-specific  plan.  In  addition,  they  spent  four  days  on 
preparation  time  and  travel  to  and  from  the  plants.  Further  time  was  spent  talking  to  plant  personnel  on 
the  telephone  about  the  plan. 

The  plant  quality  control  manager  and  staff  were  usually  responsible  for  preparing  the  HACCP  plant- 
specific  plan.  Other  plant  management  were  involved  in  its  preparation  at  some  plants.  In  addition  to  the 
time  spent  with  the  HACCP  Special  Team,  plants  spent  two  to  twenty-four  person-days  developing  their 
plan  and  familiarizing  plant  employees  with  HACCP. 

Plants  whose  personnel  were  familiar  with  HACCP  were  able  to  develop  a  plant-specific  plan  more 
quickly.  Those  to  whom  the  process  was  unfamiliar  would  need  an  increased  amount  of  information  and 
training  to  develop  a  plan  on  their  own. 


*  Hazard  Analysis  Critical  Control  Points  (HACCP)  is  a  process  control  system  that  identifies  "critical"  points  in  the  food  production 
process  that  should  be  controlled  to  prevent  food  safety  hazards  from  occurring. 
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I.  INTRODUCTION 


Purpose 

To  describe  the  activities  undertaken  by  the  HACCP*  Special  Team  and  pilot  plant  personnel  during  on- 
site  visits  to  the  plants  and  during  the  development  of  a  plant-specific  plan  using  the  generic  plan 
developed  at  the  HACCP  workshop  as  a  model. 


Background 

In  January  1990,  the  Food  Safety  and  Inspection  Service  (FSIS)  began  a  study  to  determine  the  process 
for  Hazard  Analysis  Critical  Control  Points  (HACCP)  implementation  in  meat  and  poultry  inspection 
operations.  As  part  of  the  study,  five  workshops  were  held  with  industry  to  develop  model  HACCP  plans 
for  specific  products  and  processes.  Those  chosen  included  Refrigerated  Foods,  Cooked  Sausage,  Poultry 
Slaughter,  Ground  Beef,  and  Swine  Slaughter. 

Following  the  Refrigerated  Foods,  Cooked  Sausage,  and  Poultry  Slaughter  workshops,  the  generic  HACCP 
models  developed  at  these  workshops  were  piloted  in  three  volunteer  plants  for  each  product/process.  An 
on-site  visit  at  each  of  the  pilot  facilities  was  conducted  by  two  members  of  the  HACCP  Special  Team 
assigned  to  that  plant.  This  visit  began  the  process  of  matching  a  plant-specific  HACCP  plan  to  the 
generic  HACCP  model  developed  at  the  workshop.  The  two-member  team,  using  a  "model  checklist" 
developed  by  FSIS  to  ensure  that  all  HACCP  principles  would  be  met,  continued  to  work  with  the  plant 
until  the  plant-specific  plan  was  complete. 


Methodology 

The  Policy  Analysis  Unit  (PAU),  FSIS,  as  part  of  the  HACCP  study,  examined  on-site  visits  to  pilot  plants 
and  the  development  of  plant-specific  plans  from  generic  models  developed  at  the  workshops.  PAU 
developed  a  series  of  questionnaires  to  assess  the  time  and  effort  the  HACCP  Special  Team  and  pilot  plant 
management  spent  developing  plant-specific  plans.  A  PAU  analyst  administered  the  questionnaires  either 
in  person  or  by  telephone  with  HACCP  Special  Team  members.  HACCP  Special  Team  members 
administered  the  questionnaires  to  plant  management  in  the  volunteer  plants. 

•     The  first  questionnaire  was  designed  to  measure  Special  Team  activities  during  the  on-site 
visit. 


The  second  questionnaire  was  designed  to  assess  the  time  and  effort  the  pilot  plant  spent  on 
this  activity  -  outside  time  spent  with  the  Special  Team  -  by  the  end  of  the  on-site  visit. 


*  Hazard  Analysis  Critical  Control  Points  (HACCP)  is  a  process  control  system  that  identifies  "critical"  points  in  the  food  production 
process  that  should  be  controlled  to  prevent  food  safety  hazards  from  occurring. 


The  third  questionnaire  was  administered  to  the  Special  Team  at  the  completion  of  plant- 
specific  plan  development  to  gain  additional  information  about  their  activities  in  plant-specific 
plan  development. 

A  fourth  questionnaire  was  administered  to  plant  management  at  the  completion  of  plant 
specific  plan  development  to  gain  additional  information  about  their  activities  in  plant  specific 
plan  development. 

Both  Special  Team  members  and  plant  staff  were  asked  to  report,  the  amount  of  time  they  spent  on 
activities  to  develop  the  plant-specific  plan.  Their  answers  were  often  vague  and  incomplete.  Therefore, 
some  times  estimated  in  the  report  are  based  on  the  following  assumptions: 

All  time  spent  at  the  initial  on-site  by  Special  Team  members  and  plant  personnel  whether  it  was 
familiarizing  FSIS  inspectors  and  plant  personnel  with  HACCP,  developing  the  plant-specific 
model,  touring  the  plant  to  match  critical  control  points  (CCPs)  from  the  generic  plan  to  the  plant, 
or  actually  working  with  the  plant-specific  model  was  counted  as  time  spent  developing  the  plant- 
specific  model.  All  these  activities  were  necessary  for  plan  development. 

Special  Team  members  did  not  isolate  the  amount  of  time  they  had  spent  preparing  for  an  on-site 
visit  from  other  HACCP  activities,  nor  did  they  indicate  when  a  travel  day  did  not  include  a  full 
day  of  travel.  For  this  report  the  two  activities  are  combined  by  counting  each  travel  day  as  a 
preparation/travel  day. 

The  report  includes  only  the  time  spent  developing  the  plant-specific  plans  during  the  baseline 
phase  of  the  pilot  tests,  It  does  not  include  time  spent  modifying  the  plant-specific  plans  during 
the  implementation  phase. 


Report  Plan 

This  report  begins  with  a  chapter  describing  the  on-site  visits  and  development  of  plant-specific  plans  for 
the  three  Refrigerated  Foods  pilot  plants.  The  next  chapter  reports  on  the  on-site  visits  and  development 
of  plant-specific  plans  for  the  three  Cooked  Sausage  pilot  plants  The  third  chapter  describes  on-site  visits 
and  development  of  plant-specific  plans  for  the  three  Poultry  Slaughter  pilot  plants.  Detailed  responses 
for  each  of  the  nine  pilot  plants  keyed  to  the  questionnaires  are  found  in  the  Appendices.  The  report 
concludes  with  a  overall  summary  of  the  on-site  visits  and  development  of  plant-specific  plans. 
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II.  REFRIGERATED  FOODS  PILOT  PLANTS 


Introduction 

This  chapter  briefly  describes  each  of  three  Refrigerated  Foods  plants'  experience  with  the  on-site  visit 
and  plant-specific  plan  development.  Each  of  the  Refrigerated  Foods  plants  were  very  different  from  one 
another  in  size,  plant  familiarity  with  HACCP,  and  specific  product  for  the  HACCP  pilot.  (See  Appendix 
B  for  responses  to  all  Refrigerated  Foods  questionnaires.) 


Refrigerated  Foods  -  1 

This  large  chicken  processing  plant  had  already  begun  using  HACCP  type  controls  on  its  product,  fully 
cooked  chicken,  before  volunteering  for  the  pilot  project.  The  first  on-site  visit  was  held  at  this  plant  on 
June  10-11,  1991.  The  entire  Special  Team  attended  this  session  along  with  regional  and  area  personnel. 
HACCP  Special  Team  members  used  the  model  checklist  to  compare  the  generic  plan  to  the  plant-specific 
model,  toured  the  plant  and  met  with  plant  officials,  including  the  quality  control  manager. 

The  plant  management  did  not  want  to  commit  to  a  plant-specific  plan  at  this  meeting.  They  were 
intimidated  by  all  the  FSIS  personnel  who  attended.  They  were  concerned  that  any  specific  controls  found 
in  their  plan  would  be  turned  into  regulations.  The  two  HACCP  Special  Team  members  assigned  to  the 
plant  returned  later  for  another  visit  to  finish  the  plant-specific  plan. 

The  two  Special  Team  members  each  spent  four  days  at  the  plant,  four  days  of  travel  and  preparation  time, 
and  approximately  eight  hours  on  the  telephone  with  the  plant  for  a  total  of  approximately  17  days. 

In  addition  to  the  four  days  spent  with  the  Special  Team  at  the  on-site  and  later  visit,  the  plant 
management  spent  time  preparing  the  plan  and  four  hours  providing  training  to  their  staff. 

The  plant-specific  plan  for  this  plant  was  completed  October  10,  1991. 


Refrigerated  Foods  -  2 

This  small  plant  used  a  different  product  flow  from  the  generic  model.  The  generic  model  was  designed 
for  a  cooked  and  assembled  product.  This  product,  chicken  mix  in  sauce,  was  assembled  into  a  cooking 
vat,  cooked  and  poured  hot  into  bags.  However,  Special  Team  members  determined  that  the  differences 
were  not  enough  to  warrant  changing  the  generic  plan.  This  plant  was  a  Total  Quality  Control  (TQC) 
plant  so  the  staff  utilized  their  TQC  programs  to  design  the  HACCP  plan. 

The  on-site  visit  at  this  plant  was  held  July  23-24, 1991.  Special  Team  members  used  the  model  checklist, 
toured  the  plant,  and  met  with  plant  management  and  the  inspector-in-charge  (HQ. 


The  two  Special  Team  members  spent  two  days  at  the  on-site,  one  Special  Team  member  spent  an 
additional  two  days  at  the  plant,  four  days  each  were  spent  on  travel  and  preparation  time,  and  time  was 
spent  on  the  telephone  with  the  plant  for  a  total  of  approximately  16  days. 
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In  addition  to  the  time  spent  with  the  Special  Team,  the  plant  speut  three  days  developing  the  plant 
specific  plan  and  preparing  for  meeting  with  the  Special  Team 


The  plant-specific  plan  for  this  plant  was  completed  October  30,  199! 


Refrigerated  Foods  -  3 

The  product  involved  in  the  HACCP  pilot  for  this  medium  size  plant,  a  beef  and  bean  burito,  was  not 
made  every  day  so  the  amount  of  time  needed  to  collect  baseline  data  took  longer  than  planned.  The  plant 
had  been  a  problem  plant  several  years  ago,  but  it  had  acquired  a  new  owner  who  was  interested  in 
participating  in  the  pilot  and  had  a  quality  control  manager  who  was  enthusiastic  about  the  program. 

An  on-site  visit  at  trie  plant  was  held  August  12-14,  1991.  Special  ream  members  held  an  awareness 
session  and  toured  the  plant  to  compare  the  process  of  the  product  chosen  for  the  HACCP  pilot  with  the 
industry  workshop  model.  Four  plant  employees  attended  the  awareness  training.  Their  HACCP  plan  was 
submitted  twice,  but  it  was  returned  for  more  specificity  A  Special  Team  member  visited  the  plant  on  i 
a  regular  basis  to  work  with  the  quality  control,  manager. 

The  two  Special  Team  members  assigned  to  this  plant  spent  three  days  at  the  on-site,  an  additional  11 
hours  were  spent  at  other  plant  visits,  phone  calls  with  the  plant  totaled  another  seven  hours  and  four  days 
of  travel  and  preparation  time  were  spent  for  a  total  of  approximately  16-17  days. 

In  addition  to  the  time  spent  during  Special  Team  visits  and  phone  calls,  the  plant  spent  three  days  on 
plant-specific  plan  development 


Fhe  plant-specific  plan  for  this  plant  was  completed  January  9,  1992. 


Summary 

AH  ihree  pilot  plants  included  "receiving51  as  a  CCP  in  their  plant-specific  plan  although  it  was  not 
included  in  the  generic  model  developed  at  the  workshop.  As  a  result,  the  HACCP  Special  Team 
contacted  the  Refrigerated  Foods  Steering  Committee  to  incorporate  "receiving"  as  a  CCP  in  the  generic 
model 

The  process  of  developing  plant-specif  ic  plans  for  these  plants  took  from  three  to  five  months  from  the 
date  of  the  initial  on-site  visits  until  the  plans  were  completed. 


III.  COOKED  SAUSAGE  PILOT  PLANTS 


Introduction 


This  chapter  briefly  describes  each  of  the  three  Cooked  Sausage  plants'  experience  with  the  on-site  visit 
and  plant-specific  plan  development.  (See  Appendix  C  for  responses  to  all  Cooked  Sausage 
questionnaires.) 

Cooked  Sausage  -  1 

The  on-site  visit  for  this  family  owned  and  operated  plant  took  place  October  15-17,  1991.  The  Special 
Team  used  the  model  checklist  to  compare  plant  processes  for  producing  its  HACCP  product,  a  meat 
frankfurter,  with  the  generic  model,  toured  the  plant,  and  met  with  plant  management  and  the  EC.  The 
plant-specific  plan  was  returned  to  the  plant  after  its  first  submission  for  more  specificity.  FSIS  personnel 
reviewing  the  plan  indicated  that  the  plant  needed  to  designate  who  in  the  plant  would  perform  monitoring 
and  verification  tasks  at  each  CCP. 

The  two  Special  Team  members  assigned  to  this  plant  spent  three  days  at  the  one-site,  two  days  at  a 
second  visit,  a  day  making  phone  calls  and  four  days  in  travel  and  preparation  time  for  a  total  of 
approximately  19  days. 

In  addition  to  the  time  spent  with  the  Special  Team,  plant  management  said  they  had  spent  24  days 
writing,  typing,  and  modifying  the  plant-specific  plan. 

The  plant-specific  plan  for  this  plant  was  completed  May  4,  1992. 
Cooked  Sausage  -  2 

The  on-site  visit  at  the  plant  took  place  on  January  21-23,  1992.  The  Special  Team  toured  the  plant  to  see 
if  the  process  for  producing  their  product,  cooked  skinless  frankfurters,  was  applicable  and  met  with  plant 
management  The  plant  was  not  familiar  with  HACCP  and  had  just  begun  a  TQC  program  so  it  took 
some  time  to  help  them  design  their  HACCP  plan. 

Data  collection  at  this  small  plant  took  longer  than  planned  as  the  HACCP  product  was  not  produced 
every  day. 

The  two  Special  Team  members  assigned  to  this  plant  spent  three  days  at  the  on-site,  two  days  at  a  second 
visit,  and  four  days  in  travel  and  preparation  time  for  a  total  of  approximately  18  days. 

A  new  quality  control  manager  was  hired  for  this  plant  during  the  time  the  plant-specific  plan  was  being 
prepared.  The  new  manager  said  he  spent  30  hours  learning  about  HACCP  and  30  hours  writing  the  plan. 
The  implementation  phase  began  before  the  final  plant-specific  plan  was  approved. 

The  plant- specific  plan  for  this  plant  was  completed  at  the  end  of  January,  1993. 


Cooked  Sausage  -  3 

The  on-site  visit  at  this  plant  was  held  in  two  sessions,  April  22-23,  and  April  27-29,  1992,  because  the 
plant  was  small  and  the  employees  who  needed  to  attend  the  on-site  could  not  all  be  spared  at  one  time. 
Their  HACCP  product  was  a  frankfurter  in  natural  casing.  The  first  meeting  was  attended  by  operations 
staff  and  the  second  by  management.  When  the  Special  Team  shared  the  results  of  the  two  groups,  plant 
management  were  surprised  by  the  differences  in  their  perceptions  of  HACCP.  That  led  to  a  good 
discussion  between  the  groups. 

The  two  Special  Team  members  assigned  to  this  plant  spent  six  days  at  the  on-site,  two  days  at  a  second 
yisit,  and  four  days  in  travel  and'  preparation  time  for  a  total  of  approximately  20  days. 

In  addition  to  time  spent  with  the  Special  Team,  plant  owner  estimated  20  hours  had  been  spent  preparing 
their  plant-specific  plan.  The  quality  control  manager  who  had  written  the  plant-specific  plan  left  the 
company  so  revisions  were  made  by  the  company  president 

The  plant-specific  plan  for  this  plant  was  completed  November  2,  1992 


Summary 

Following  their  experience  with  the  Refrigerated  Food  pilot  plants,  Special  Team  members  found  it  useful 
to  have  the  HACCP  quantitative  evaluation  task  data  collector  attend  the  on-site  visit  at  the  pilot  plants. 
In  this  way  all  people  at  the  plant  connected  with  the  HACCP  pilot  were  involved  from  the  beginning  and 
understood  the  reasons  for  each  other's  activities. 


Plants  took  from  six  months  to  a  year  to  complete  their  plant-specific  plans. 
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IV.  POULTRY  SLAUGHTER  PILOT  PLANTS 


Introduction 


This  chapter  briefly  describes  each  of  the  three  Poultry  Slaughter  plants'  experience  with  the  on-site  visit 
and  plant-specific  development.  These  were  the  first  plants  to  pilot  test  HACCP  slaughter  operations,  as 
the  other  six  HACCP  pilots  were  tests  of  processing  operations.  (See  Appendix  D  for  responses  to  all 
Poultry  Slaughter  questionnaires.) 

All  on-site  visits  for  Poultry  Slaughter  pilot  plants  were  held  at  the  same  time,  May  26-29,  1992. 


Poultry  Slaughter  -  1 

About  35  people  attended  the  on-site  visit  at  this  large  New  Line  Speed  (NELS)  plant.  The  Special  Team 
gave  a  HACCP  awareness  presentation,  toured  the  plant  (in  two  groups)  and  matched  the  plant  process 
for  slaughtering  whole  young  chickens  to  the  generic  model.  No  one  from  this  plant  had  attended  the 
Poultry  Slaughter  workshop  so  they  had  had  no  prior  HACCP  exposure.  The  entire  plant  slaughter  process 
was  under  the  HACCP  pilot  as  all  lines  inhabit  a  common  area  during  the  process.  Their  plant  was 
already  using  some  additional  food  safety  controlling  steps  not  included  in  the  generic  model.  The  plant 
included  these  steps  in  their  model. 

The  two  Special  Team  members  each  spent  four  days  at  the  plant,  four  days  of  travel  and  preparation  time, 
and  a  number  of  hours  on  the  telephone  with  the  plant  personnel  for  a  total  of  approximately  17  days. 

In  addition  to  the  four  days  spent  with  the  Special  Team  at  the  on-site  and  later  visit,  the  plant 
management  spent  six  hours  on  meetings  with  staff  and  66  hours  developing  the  plan. 

The  plant- specific  plan  for  this  plant  was  completed  July  30,  1992. 
Poultry  Slaughter  -  2 

The  Special  Team  met  with  FSIS  inspectors  and  plant  management  to  provide  an  introduction  to  HACCP, 
toured  the  plant  and  matched  the  generic  plan  to  the  plant  process  for  slaughtering  whole  young  chickens. 
This  was  the  first  plant  to  have  developed  a  draft  plant-specific  plan  prior  to  the  on-site  visit.  They  had 
had  help  from  the  quality  control  staff  at  another  pilot  plant  which  is  owned  by  the  same  company.  Their 
plant-specific  plan  did  not  contain  all  the  steps  in  the  generic  model  (i.e.  packaging  and  storage). 
However,  this  was  not  a  problem  as  this  plant  does  not  include  those  steps  in  its  operations.  Their  entire 
plant  operation  was  under  the  HACCP  plan  as  their  three  lines  share  two  chiller  tanks. 

The  Special  Team  members  each  spent  four  days  at  the  plant,  four  days  of  travel  and  preparation  time, 
and  a  number  of  hours  on  the  telephone  with  plant  management  for  a  total  of  approximately  17  days. 

In  addition  to  the  four  days  spent  with  the  Special  Team  at  the  on-site  and  later  visit,  the  plant 
management  spent  half  a  day  on  telephone  calls  and  two  days  developing  the  plant-specific  plan  and 
meeting  with  other  plant  personnel. 


The  plant- specific  plan  for  this  plant  was  completed  May  27,  1992 


Poultry  j>laujghterjJ3 

This  plant  was  selected  by  its  corporate  management  to  participate  as  a  HACCP  pilot  plant.  However, 
no  one  had  explained  to  plant  staff  what  would  be  involved  in  participating  as  a  pilot  plant.  During  the 
on-site  visit  when  the  Special  Team  started  going  over  the  requirements  for  collecting  sample  birds,  the 
plant  wanted  to  end  their  participation,  Eventually,  however,  the  plant  decided  to  continue  participation. 

The  Special  Team  did  continue  with  the  usual  procedures  holding  an  awareness  session,  touring  the  plant, 
and  matching  the  generic  model  to  the  plant  process  for  slaughtering  whole  young  chickens. 

The  Special  Team  each  spent  four  days  at  the  plant,  four  days  of  travel  and  preparation  time,  and  a 
number  of  hours  on  the  telephone  with  plant  management  for  a  total  of  approximately  17  days, 

In  addition  to  time  spent  with  the  Special  Team,  plant  management  spent  50  hours  preparing  their  plant- 
specific  plan. 

The  plant-specific  plan  for  this  plant  was  completed  October  10,  1992. 


Summary 

Since  PB1S  tasks  have  not  been  developed  for  slaughter,  new  tasks  and  recordkeeping  methods  had  to  be 
established  by  FSIS  to  monitor  the  progress  of  these  pilot  plants. 

In  addition  to  completing  their  regular  procedure  at  the  pilot  plant  on-site  visit  ai  each  Poultry  Slaughter 
pilot  plant,  the  Special  Team  held  awareness  sessions  to  explain  the  HACCP  study  to  FSIS  on-line 
inspectors  and  plant  personnel  who  wished  to  attend.  Provisions  were  made  to  grant  administrative 
overtime  for  inspectors  who  attendee]  Since  the  sessions  were  so  well  received  and  well  attended,  the 
Special  Team  planned  to  hold  Information  sessions  for  the  inspectors  during  in-plant  training  as  well 

Plants  spent  one  to  five  months  to  complete  their  plant-specific,  plans. 


V.  SUMMARY  FINDINGS 


Introduction 


This  chapter  provides  an  overall  summary  and  description  of  the  on-site  visits  and  development  of  plant- 
specific  plans.  It  describes  changes  made  to  the  original  HACCP  design,  the  applicability  of  generic 
models  to  plant-specific  plans,  the  time  and  effort  of  both  Special  Team  members  and  plant  management 
on  the  adaption  process,  and  general  findings  about  the  adaption  process. 


Changes  to  Original  Design 
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The  original  HACCP  design  for  developing  plant-specific  plans  envisioned  that  a  plant  would  develop  its 
plant-specific  plan  prior  to  an  on-site  visit  of  HACCP  Special  Team  members.  The  plant-specific  plan 
would  then  be  compared  with  the  generic  model  developed  at  the  workshop  during  the  Special  Team  visit 
at  the  pilot  plant  using  the  model  checklist  developed  by  FSIS. 

In  general,  plants  did  not  have  plant-specific  plans  ready  for  review  at  the  on-site  visit.  The  original  design 
evolved  into  a  lengthier  process  as  it  became  obvious  that  more  time  than  a  three-day  meeting  was 
necessary  to  develop  a  plant-specific  HACCP  plan.  The  initial  visit  became  an  introductory  HACCP 
session  at  the  plant  During  the  visit,  Special  Team  members  observed  how  the  product/process  was 
prepared.  In  addition,  plant  management  learned  the  process  of  developing  a  HACCP  plan  and  worked 
through  a  few  CCPs  with  Special  Team  members. 

After  the  on-site  visit,  the  HACCP  data  collector  began  collecting  baseline  data  for  the  study  while  plant 
management  completed  the  plant-specific  plan  for  FSIS  review  with  help  from  the  Special  Team  members 
assigned  to  their  plant.  The  Special  Team  considered  the  entire  three  months  of  baseline  data  collection 
as  part  of  the  on-site/plant-specific-plan  development  period.  The  Special  Team  regarded  the  process  as 
one  of  gradual  revision  over  a  period  of  time. 

In  order  for  a  plant-specific  plan  to  be  approved,  all  the  questions  on  the  HACCP  checklist  must  have 
been  answered  in  the  affirmative  for  the  plan.  A  plan  could  contain  a  few  "No"  responses  at  the  start  of 
the  implementation  phase  at  the  pilot  plant,  but  it  must  have  contained  all  "Yes"  responses  before  the 
operational  phase  could  begin.  This  process  took  up  to  one  year,  but  averaged  five  months  after  an  initial 
on-site  visit. 


Applicability  of  Generic  Model  to  Plant-Specific  Plans 

In  general,  generic  models  developed  at  the  workshops  fitted  well  with  actual  processes  in  the  pilot  plants. 
Only  one  generic  model  appeared  to  need  any  modification  following  plant-specific  plan  development 

All  three  Refrigerated  Foods  pilot  plants  included  "receiving"  as  a  CCP  in  their  plant-specific  plans.  The 
Special  Team  concluded  that  "receiving"  should  be  included  as  a  CCP  in  the  generic  model.  The  steering 
committee  coordinator  on  the  Special  Team  contacted  the  Refrigerated  Foods  Steering  Committee  to  add 
"receiving"  as  a  CCP  to  the  Refrigerated  Foods  generic  plan. 
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Both  the  Cooked  Sausage  and  Poultry  Slaughter  generic  models  were  applicable  to  the  pilot  plants.  In 
some  cases,  pilot  plants'  flowcharts  were  in  slightly  different  order  from  the  generic  model,  but  all  CCPs 
found  in  the  generic  model  were  included  in  the  plant-specific  plans,  if  those  processing  steps  were  part 
of  a  plant's  process.  I 

Time  and  Effort  of  the  HACCP  Special  Team 

Prior  to  the  on-site,  Special  Team  members  prepared  HACCP  familiarization  materials.  They  presented 
these  during  the  on-site  visit  in  addition  to  touring  the  plant  to  compare  the  generic  model  CCPs  to  the 
HACCP  product  and  meeting  with  plant  management.  Special  Team  members  generally  made  another 
visit  to  each  of  their  plants  as  part  of  the  development  process.  In  addition,  calls  were  made  between  the 
team  and  plant  management  to  answer  questions  about  the  HACCP  study  and  the  plant-specific  plan. 
Special  Team  members  worked  over  several  months  with  each  plant  management  to  bring  the  plant- 
specific  plans  up  to  acceptable  levels.  I 

The  total  amount  of  time  the  Special  Team  spent  helping  plants  develop  their  plant-specific  plans  averaged 
18  days.  This  included  four  travel  days,  four  or  five  days  at  the  plant  and  a  half  day  on  the  telephone  for 
each  of  the  two  members. 


Time  and  Effort  of  Plant  Management 

The  quality  control  manager  of  the  plant  was  usually  responsible  for  developing  the  HACCP  plant-specific 
plaa  The  quality  control  manager  was  aided  in  the  development  of  the  plan  by  the  quality  control  staff ! 
and/or  other  plant  management  including  in  some  plants  the  executive  chef,  laboratory  staff,  and  others. 
The  amount  of  time  needed  for  plan  development  depended  in  pail:  on  how  familiar  the  quality  control  I 
manager  was  with  HACCP,  PLANT,  Partial  Quality  Control  (PQC),  or  other  process  control  systems. 
Development  went  more  quickly  for  those  familiar  with  these  concepts.  Plan  preparers  had  most  difficulty 
performing  a  hazard  analysis  of  the  process  and  being  specific  enough  about  who  was  going  to  perform 
monitoring  and  evaluation  tasks  at  each  CCP, 

The  amount  of  time  spent  by  plant  personnel  in  developing  their  plant-specific  plan  ranged  from  three  to 
30  days  in  addition  to  the  approximately  four  days  spent  with  the  HACCP  Special  Team  to  develop  the 
plan.  It  included  time  spent  developing  the  plan  and  other  related  activities  such  as  familiarizing  plant 
staff  with  the  plan.  Several  quality  control  managers  said  that  the  next  HACCP  plan  they  prepared  would 
be  much  easier  to  do  as  they  would  know  how  to  go  about  it:  what  sorts  of  hazards  to  identify,  what  sorts 
of  controls  to  use,  and  how  to  monitor  the  plan. 

in  addition  to  their  quality  control  staff,  plants  also  sent  other  management  people  to  the  on-site: 
orientation.  As  many  as  ten  people  attended  at  some  plants  representing  up  to  30  days  of  employee  time. 
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Summary  Findings: 


I.  The  on-site  visit  became  a  HACCP  orientation  session  and  an  instruction  session  on  how  to  do 
a  hazard  analysis  and  develop  CCPs.  This  was  especially  important  for  those  plants  where  no 
one  had  attended  a  HACCP  workshop  and  no  one  was  familiar  with  HACCP. 

The  generic  models  developed  at  the  workshops  were  applicable  to  the  plants.  Only  in  the  case 
of  Refrigerated  Foods  was  it  necessary  to  consider  alteration  of  the  generic  model. 

It  was  important  to  have  the  HACCP  data  collector  attend  the  on-site  session.  In  that  way, 
everyone  involved  in  the  pilot  understood  the  purpose  of  the  HACCP  study  and  was  aware  of  all 
participants'  roles. 

The  time  it  took  a  plant  to  develop  a  plant-specific  plan  depended  on  its  familiarity  with  HACCP, 
PLANT,  and  similar  programs.  Some  plants  were  already  using  some  HACCP  type  controls  while 
others  had  no  knowledge  of  HACCP  and  were  only  beginning  to  be  aware  of  PLANT  programs. 
Plant  personnel  said  that  the  next  HACCP  plan  they  prepared  would  be  much  easier  to  do  after 
their  experience  with  the  first  one.  The  hazard  analysis,  which  was  the  most  difficult  aspect  of 
the  development  for  plants,  became  easier  with  the  changes  made  by  the  National  Advisory 
Committee  on  Microbiological  Criteria  for  Food  in  Principle  #1  of  the  HACCP  Principles  for 
Food  Production. 
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APPENDIX  A 


HACCP  SPECIAL  TEAM  PILOT  PLANT  ASSIGNMENTS 

Special  Team  Director:  Dr.  Dorothy  Stringfellow 

Refrigerated  Foods  Plants 


RF-1  Robert  Savage/Alice  Johnson 

RF-2  Nathaniel  Clark/John  Carlson 

RF-3  Walter  McEvers/Robert  Baele 

Cooked  Sausage 

CS-1  Nathaniel  Clark/John  Carlson 

CS-2  Robert  Savage/Alice  Johnson 

CS-3  Walter  McEvers/Robert  Baele 

Poultry  Slaughter 

PS-1  Nathaniel  Qark/John  Carlson 

PS-2  Robert  Savage/Alice  Johnson 

PS-3  Walter  McEvers/Robert  Baele 


APPENDIX  B 


RESPONSES  TO  REFRIGERATED  FOODS 
QUESTIONNAIRE 

Refrigerated  Foods  -  1 
Refrigerated  Foods  -  2 
Refrigerated  Foods  -  3 


Policy  Analysis  Unit 
February  1994 


Refrigerated  Foods  - 1 


Questionnaire  1A:  This  questionnaire  was  designed  to  measure  Special  Team  activities  during  the  On- 
Site  Visit 


PART  1:    Applicability  of  Generic  HACCP  Model 

Q-l.  What  activities  did  you  perform  to  determine  the  applicability  of  the  Generic  HACCP  Model  to  this  plant? 
Please  specify  activity  {e.g.,  used  checklists,  toured  plant,  met  with  various  people,  etc.)  and  the  amount  of 
time  expended. 

Special  Team  members  used  the  checklists,  toured  the  plant,  and  met  with  plant  officials  including 
the  Quality  Control  Manager.  The  activities  went  well.  'It  is  a  good  plant" 

The  entire  Special  Team  met  with  plant  management  for  two  days,  June  10-11, 1991. 

Q-2.  Based  on  the  On-Site  Visit,  was  the  generic  model  applicable  to  this  plant? 

__X__  Yes,  Completely  Applicable    Yes,  Somewhat  Applicable   No,  Not  At  All  Applicable 

(Skip  to  PART  2)  (Continue  with  Q-3)  (Continue  with  Q-3) 

Q-3.  What  were  the  difficulties /problems  encountered? 

No  difficulties. 

Q-4.  If  you  marked  Somewhat  or  No  in  Q-2,  whom  did  you  contact  about  the  difficulties /problems? 

HACCP  Special  Team  On-Site  Coordinator 
 HACCP  Special  Team  Director 

 Other  HACCP  Special  Team  MembeKs):  (name(s))_  

Other  FSIS  Personnel  (HQ  Area,  Regional,  personnel,  etc.) 
 Plant  Personnel  (Management,  QC /Production,  etc.) 

Other:  (title(s)/name(s))_  .  —  

Q-5.  What  activities  were  undertaken  to  remedy  these  difficulties /problems  (meetings,  phone  calls,  etc.)? 
Please  specify  the  types  and  extent  of  contact  with  each  of  the  people/groups  marked  in  Q-4. 

N/A 

Q-6.  To  address  the  difficulties  noted  in  Q-3,  did  the  pilot  plant  make  any  changes  to  their  processes/ 
procedures  to  enable  them  to  better  fit  the  Generic  HACCP  Model? 

 Yes    __X_No  (Continue  with  Q-7) 

Q-7.  To  address  the  difficulties  noted  in  Q-3,  were  any  changes  made  to  the  Generic  HACCP  Model? 

 Yes    _X_No  (Continue  with  Q-8) 

Q-8.  By  what  date  were  all  changes  incorporated  into  the  generic  model?  .  
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The  HACCP  Special  Team  requested  that  the  Refrigerated  Foods  Steering  Committee  include 
"receiving"  as  a  CCP  in  the  Refrigerated  Foods  generic  model.  To  date  the  committee  has  not  met  to 
make  the  change, 


PART  2:     Plant-Specific  HACCP  Plan 

Q-9,  Did  the  pilot  plant  have  an  acceptable  plant-spedrk  plan  in  place  at  the  start  of  the  On-Site  Visit? 


Q-10. If  you  marked  Partially  Acceptable  or  No/Unacceptable  in  Q-9,  what  did  you  do  to  develop  or  initiate 
development  of  a  plant-specific  HACCP  plan  (meetings/ phone  calls,  etc.:  how  many,  how  long, 
with  whom,  etc.)? 

At  the  initial  meeting,  the  plant  didn't  want  to  commit  to  a  specific  plan  in  front  of  twenty  or  more 
FSIS  staff.  Plant  management  was  also  fearful  of  any  commitment  being  turned  into  regulations. 

Q-ll.Was  a  plant-specific  HACCP  plan  completed  by  the  end  of  the  'On-Site  Visit? 


If  no,  please  specify  what  remains  to  be  done  and  when  do  you  anticipate  such  a  plan  to  be  completed  I 
ready  for  implementation. 

Two  Special  Team  members  returned  a  few  weeks  later  to  work  with  the  plant  on  a  plant 
specific  plan.  They  met  with  the  plant  for  two  days.  The  plant-specific  plan  was  completed 
October  10, 1991, 

Q-12.  Were  there  any  other  Special  Team  activities  that  occurred  during  the  On-site  Visit  concerning 
development  of  a  plant -specific  plan  based  on  the  generic  model  that  we  haven't  discussed? 

Baseline  data  collection  began  July  22, 1991 


Yes,  Completely  Acceptable 
(Skip  to  Q-ll) 


Yes,  Partially  Acceptable 

(Continue  v/ith  Q-10) 


X__  No  PlanAJnacceptable  F 
(Continue  with  Q-10) 


Yes  (Continue  with  Q-12) 


X  No 
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Questionnaire  IB:  This  questionnaire  was  designed  to  assess  the  time  and  effort  of  pilot  plants  in  develop- 
ing the  plant-specific  plans  prior  to  and  during  the  On-Site  Visit. 


f-1.  Who  on  your  staff  worked  on  the  adaptation  of  the  generic  HACCP  model  to  a  plant-specific  HACCP  plan? 
(titles/numbers  of  people  involved,  etc.) 

Quality  Control  (QC)  Manager 

Plant  Manager 


QC  Supervisors  -  2 
Plant  Superintendent 


•-2.  What  activities  were  undertaken  to  develop  the  plant-specific  plan  (e.g.,  meetings,  phone  calls,  training 
sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics  were  covered  and  what  was 
accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was  there  preparation  time  for  the 
activities?  If  so,  how  much  time? 

Two  training  sessions  were  held  for  plant  employees.  The  first  was  a  45  minute  session  for  plant 
supervisors  to  describe  HACCP,  how  it  affects  the  plant  and  their  role.  The  second  training  session 
was  for  all  QC  technicians  to  discuss  their  duties  and  role  in  HACCP.  This  took  about  two  hours. 


•3.  Were  there  any  other  pilot  plant  activities  concerning  the  development  of  a  plant-specific  plan  based  on 
the  generic  model  that  occurred  prior  to  or  during  the  On-Site  Visit  that  we  haven't  discussed? 


No. 


— 
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Questionnaire  2A:  This  questionnaire  was  designed  for  the  Special  Team  members  at  the  completion 
the  plant-specific  plan. 


-1.  Since  the  On-Site  Visit,  what  did  you  do  to  continue  the  development  of  the  plant-specific  HACCP  plan^i 
Please  be  specific  (meetings,  phone  calls,  etc.:  how  many,  how  long,  with  whom). 

Phone  conversations  were  held  with  the  plant  about  the  pl&sit-specif  ic  plan. 


Q-2,  When  was  the  plant-specific  HACCP  plan  completed?   October  10, 1991 


Q-3.  Were  there  any  other  Special  Team  activities  that  occurred  after  the  On-Site  Visit  concerning 

development  of  a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes, 
please  specify 


No, 
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Questionnaire  2B:  This  questionnaire  was  designed  for  plant  management  at  the  completion  of  the  plant- 
specific  plan. 


Since  the  On-Site  Visit,  what  activities  were  undertaken  to  develop  the  plant-specific  model 
(e.g.,  meetings,  phone  calls,  training  sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics 
were  covered  and  what  was  accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was 
there  preparation  time  for  the  activities?  If  so,  how  much  time? 

The  plant  held  meetings  to  update  junior  plant  personnel  regarding  changes  to  the  HACCP  plan  and 
to  up-date  supervisors  on  the  HACCP  Agenda.  They  estimated  45  minutes  for  this  activity. 


Since  the  On-Site  Visit,  have  there  been  any  changes  in  the  numbers  or  types  staff  (titles)  who  worked  on 
the  development  of  a  plant-specific  HACCP  plan  based  on  the  generic  HACCP  model?  If  yes,  please 
specify. 

No. 


•3.  Were  there  any  other  pilot  plant  activities  that  occurred  after  the  On-Site  Visit  concerning  development  of 
a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes,  please  explain. 


No. 
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Questionnaire  1A:  This  questionnaire  was  designed  to  measure  Special  Team  activities  during  the  On- 
Site  Visit. 


PART  1:    Applicability  of  Generic  HACCP  Model 

Q-l.  What  activities  did  you  perform  to  determine  the  applicability  of  the  Generic  HACCP  Model  to  this  plant? 
Please  specify  activity  (e.g.,  used  checklists,  toured  plant,  met  with  various  people,  etc.)  and  the  amount  of 
time  expended. 

Special  Team  members  used  a  checklist  developed  by  the  HACCP  Special  Team  to  compare  plant 
processes  to  the  generic  model,  toured  the  plant,  and  met  with  plant  management  and  the  inspector-in- 
charge(IIC). 

The  Special  Team  met  with  plant  management  July  23-24, 1991,  to  develop  the  plant-specific  plan. 

Q-2.  Based  on  the  On-Site  Visit,  was  the  generic  model  applicable  to  this  plant? 

 Yes,  Completely  Applicable    _X      Yes,  Somewhat  Applicable   No,  Not  At  All  Applicabk 

(Skip  to  PART  2)  (Continue  with  Q-3)  (Continue  with  Q-3) 

Q-3.  What  were  the  difficulties/ problems  encountered?  Please  specify. 

There  were  some  differences  between  the  generic  model  and  this  product  process,  but  they  were  not 
sufficient  to  warrant  changing  the  generic  plan.  There  were  differences  in  how  the  product  was 
made.  The  generic  model  was  designed  for  a  cooked  and  assembled  product.  This  product  was 
assembled  into  a  cooking  vat,  cooked  and  poured  hot  into  bags. 


Q-4.  If  you  marked  Somewhat  or  No  in  Q-2,  whom  did  you  contact  about  the  difficulties/ problems? 
(Check  all  that  apply.) 

_X_HACCP  Special  Team  On-Site  Coordinator 
_X_  HACCP  Special  Team  Director 

_X_  Other  HACCP  Special  Team  Member(s):  (name(s))  Robert  Savage,  entire  Special  Team 
_X_Other  FSIS  Personnel  (HQ  Area,  Regional,  personnel,  etc.) 
_X_Plant  Personnel  (Management,  QC/Production,  etc.)Plant  manager 

 Other:  (title(s)/name(s))  


Q-5.  What  activities  were  undertaken  to  remedy  these  difficulties /problems  (meetings,  phone  calls,  etc.)? 
Please  specify  the  types  and  extent  of  contact  with  each  of  the  people/groups  marked  in  Q-4. 

A  phone  call  was  made  to  Dr.  Leary,  Special  Team  Director,  and  Robert  Savage  to  determine  if  changes 
in  the  generic  model  needed  to  be  made  because  of  the  difference  in  product  flow. 

Conference  call  with  Special  Team  Director  July  24, 1991.  Approximate  time  30  minutes. 
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.1, 


Q-6.  To  address  the  difficulties  noted  in  Q-3,  did  the  pilot  plant  make  any  changes  to  their  processes/ 
procedures  to  enable  them  to  better  fit  the  Generic  HACCP  Model? 

 Yes     __X_No  (Continue  with  Q 

Q-7.  To  address  the  difficulties  noted  in  Q-3,  were  any  changes  made  to  the  Generic  HACCP  Model? 

_Yes     _X_No  (Continue  with  ( 

Recommendations  were  made  through  Special  Team  Coordinator  to  Steering  Committee 
for  "receiving"  to  be  added  to  the  generic  model  as  a  critical  control  point  (CCPK 

Q-8.  By  what  date  were  all  changes  incorporated  into  the  generic  model?  

To  date  the  generic  model  has  not  been  revised, 

PART  2:     Plant  Specific  HACCP  Plan 

Q-9.  Did  the  pilot  plant  have  an  acceptable  plant-specific  plan  in  place  at  the  start  of  the  On-Site  Visit? 

 Yes,  Completely  Acceptable   __X  Yes,  Partially  Acceptable   No  Plan/Unacceptable  1 

(Skip  to  Q-11)  (Continue  with  Q-10)  (Continue  with  Q-10) 

Q-10. If  you  marked  Partially  Acceptable  or  No/Unacceptable  in  Q-9,  what  did  you  do  to  develop  or  initiate 
development  of  a  plant-specific  HACCP  plan  (meetings /phone  calls,  etc.:  how  many,  how  long, 
with  whom,  etc )? 

Their  flow  chart  needed  to  be  changed  as  it  included  some  non-safety  issues.  This  wa?  pointed 
out  to  the  plant  management. 

A  combination  of  meetings  and  telephone  conversations  with  the  IIC  and  plant  managment  were 
held  to  develop  the  plant-specific  plan, 
(See  Questionnaire  2A.) 


Q-11.  Was  a  plant-specific  HACCP  plan  completed  by  the  end  of  the  On-Site  Visit? 

 Yes  (Continue  with  Q-12)  ._X_No 

If  no,  please.1  specify  what  remains  to  be  done  and  when  you  anticipate  such  a  plan  to  be  completed  an] 
ready  for  implementation 

Special  Team  members  reviewed  thoroughly  the  first  two  CCPs  with  plant  management  and  were 
satisfied  that  plant  management  could  finish  the  rest  of  the  CCPs  during  the  baseline  data  collectioi 
phase  of  the  pilot. 

The  majority  of  t^e  plant- specific  plan  was  completed,  only  minor  revisions  were  necessary. 


Q  -12.  Were  there  any  other  Special  Team  activities  mat  occurred  during  the  On-site  Visit  concerning 

development  of  a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes, 
please  explain. 

Special  Team  members  held  a  HACCP  awareness  training  session  during  which  time  the  entire 
process  of  developing  a  HACCP  plan  was  explained  to  the  plant  management.  The  plant  staff  then 
took  their  HACCP  plan  and  looked  at  each  part  in  light  of  the  information  they  had  received. 
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Questionnaire  IB:  This  questionnaire  was  designed  to  assess  the  time  and  effort  of  pilot  plants  in  develop- 
ing the  plant-specific  plans  prior  to  and  during  the  On-Site  Visit. 


1.  Who  on  your  staff  worked  on  the  adaptation  of  the  generic  HACCP  model  to  a  plant-specific  HACCP  plan? 
(titles/numbers  of  people  involved,  etc.) 

Plant  Manager 

Quality  Control  Manager 


1  What  activities  were  undertaken  to  develop  the  plant-specific  plan  (e.g.,  meetings,  phone  calls,  training 
sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics  were  covered  and  what  was 
accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was  there  preparation  time  for  the 
activities?  If  so,  how  much  time? 


See  Questionnaire  28. 


J.  Were  there  any  other  pilot  plant  activities  concerning  the  development  of  a  plant-specific  plan  based  on 
the  generic  model  that  occurred  prior  to  or  during  the  On-Site  Visit  that  we  haven't  discussed? 
If  yes,  please  explain. 


No. 
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Questionnaire  2A:  This  questionnaire  was  designed  for  the  Special  Team  members  at  the  completion  I 
the  plant-specific  plan. 

Q-l.  Since  the  On-Site  Visit,  what  did  you  do  to  continue  the  development  of  the  plant-specific  HACCP  plaj 
Please  be  specific  (meetings,  phone  calls,  etc   how  many,  bow  long,  with  whom).  j  .1. 

Dr.  Clark  met  again  with  the  plant  September  3-5, 1991,  to  assist  in  the  development  of  the  plant-spc 
plan.  In  addition,  he  talked  to  the  p!anf,  on  the  phone  eight  times  and  Mr.  Carlson  talked  to  the  plai 
four  times, 

On  October  30, 1991,  Mr.  Carlson  and  Dr.  Clark  had  a  conference  call  with  the  plant  management  ab 
revisions  to  the  plant-specific  plan, 

Approximate  time  =  Sixteen  hours. 

Approximate  time  =  Eight  calls  %  20  minutes  per  call  - 160  minutes. 


Q-2.  When  was  the  plant-specific  HACCP  plan  completed  ?_  .October  30, 1991. 


Q-3.  Were  there  any  other  Special  Team  activities  that  occurred  after  the  On-Site  Visit  concerning 

development  of  a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes,  1^ 
please  specify. 

Data  collection  took  longer  than  the  three  months  time  period  because  the  HACCP  product  at  the  pi 
was  not  made  everyday. 
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Questionnaire  2B:  This  questionnaire  was  designed  for  plant  management  at  the  completion  of  the  plant- 
specific  plan. 


1.  Since  the  On-Site  Visit,  what  activities  were  undertaken  to  develop  the  plant-specific  model 

(e.g.,  meetings,  phone  calls,  training  sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics 
were  covered  and  what  was  accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was 
there  preparation  time  for  the  activities?  If  so,  how  much  time? 

Activity  Purpose/Topic  Accomplishments  Time  in  Activity  Preparation  Time 

Meeting  Develop  specific  plan  Refined  plan  4  days  2  days 

(Meetings  were  held  with  USDA  HACCP  Special  Team  Members  and  IIC) 

Meeting  Changes  to  plant  specific  plan  Corrected  1  day  1  day 

(Meeting  with  Special  Team  members,  Regional  Coordinator,  PIC  Area,  CS  and  HO 

Meeting  Evaluation  and  completion  of  Phase  II  Implementation  Checklist  and 

on-site  questionnaire  that  determines  the  approved  plant  specific  HACCP 
plan  and  that  all  monitoring  procedures  are  correct  and  relevant  and  according 
to  instructions  received  from  the  Special  Team. 


2.  Since  the  On-Site  Visit,  have  there  been  any  changes  in  the  numbers  or  types  staff  (titles)  who  worked  on 
the  development  of  a  plant-specific  HACCP  plan  based  on  the  generic  HACCP  model?  If  yes,  please 
specify. 

No. 


3.  Were  there  any  other  pilot  plant  activities  that  occurred  after  the  On-Site  Visit  concerning  development  of 
a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes,  please  explain. 


No. 
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Questionnaire  1A:  This  questionnaire  was  designed  to  measure  Special  Team  activities  during  the  On- 
Site  Visit. 


PART  1:    Applicability  of  Generic  HACCP  Model 

Q-l.  What  activities  did  you  perform  to  determine  the  applicability  of  the  Generic  HACCP  Model  to  this  plant? 
Please  specify  activity  (e.g.,  used  checklists,  toured  plant,  met  with  various  people,  etc.)  and  the  amount  of 
time  expended. 

Prior  to  the  awareness  On-Site,  the  Circuit  Supervisor  had  approached  the  plant  to  see  if  it  would 
consider  being  a  pilot  plant  Bobert  Baele,  Special  Team  member,  visited  with  plant  personnel. 

The  On-site  took  place  on  August  12-14, 199L  HACCP  Special  Team  members  met  with  the  FSIS 
people  attending  the  On-Site  first  and  men  met  with  the  plant  and  FSIS  people  together.  The  group 
toured  the  plant  comparing  the  generic  flowchart  developed  at  the  workshop  with  the  plant-specific 
process  itself. 

The  Special  Team  met  with  plant  management  for  3  days. 

Q-2.  Based  on  the  On-Site  Visit,  was  the  generic  model  applicable  to  this  plant? 

X  Yes,  Completely  Applicable   Yes,  Somewhat  Applicable   No,  Not  At  All  Applicable 

(Skip  to  PART  2)  (Continue  with  Q-3)  (Continue  with  Q-3) 

Q-3.  What  were  the  difficulties/problems  encountered?  Please  specify. 

I     No  difficulties. 

Q-4.  If  you  marked  Somewhat  or  No  in  Q-2,  whom  did  you  contact  about  the  difficulties/problems? 
(Check  all  that  apply.) 

HACCP  Special  Team  On-Site  Coordinator 

HACCP  Special  Team  Director 
 Other  HACCP  Special  Team  MembeKs):  (name(s))_   

Other  FSIS  Personnel  (IIC,  Area,  Regional,  personnel,  etc.) 
 Plant  Personnel  (Management,  QC /Production,  etc.) 

Other.  (title(s)/name(s))  .  

Q-5.  What  activities  were  undertaken  to  remedy  these  difficulties/problems  (meetings,  phone  calls,  etc.)? 
Please  specify  the  types  and  extent  of  contact  with  each  of  the  people/groups  marked  in  Q-4. 


N/A 


Q-6.  To  address  the  difficulties  noted  in  Q-3,  did  the  pilot  plant  make  any  changes  to  their  processes/ 
procedures  to  enable  them  to  better  fit  the  Generic  HACCP  Model? 

Yes       X   No  (Continue  with  Q-7) 
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7,  To  address  the  difficulties  noted  in  Q-3,  were  any  changes  made  to  the  Generic  HACCP  Model? 

 Yes   X  No  (Continue  with 

f "Receiving"  was  a  CCP  in  this  plant  The  Special  Team  has  requested  that  the  Steering  Committee 
to  address  adding  "receiving"  as  a  CCP  to  the  generic  model.  The  Ref  rigerated  Foods  Steering  Comr 
has  taken  no  action  as  yet 


Q-S.  By  what  dale  were  all  changes  incorporated  into  die  generic  model?  

PART  2;     Plant-Specific  HACCP  Plan 

Q-9.  Did  the  pilot  plant  have  an  acceptable  plant-specific  plan  in  place  at  the  start  of  the  On-Site  Visit? 

Yes,  Completely  Acceptable   Yes,  Partially  Acceptable       X     No  Plan/Unacceptable 

(Skip  to  Q-ll)  (Continue  with  Q-10)  (Continue  with  Q-10) 

Q-10.  If  you  marked  Partially  Acceptable  or  No/Unacceptable  in  Q-9,  what  did  you  do  to  develop  or  initiate 
development  of  a  plant -specific  HACCP  plan  (meetings /phone  calls,  etc.:  how  many,  how  long, 
with  whom,  etc.)? 


During  the  On-Site,  Special  Team  members  met  with  plant  personnel  providing  them  with  informa 
to  use  to  develop  their  plant-specific  plan, 

Q-ll. Was  a  plant-specific  HACCP  plan  completed  by  the  end  of  the  On-Site  Visit? 

 Yes  (Continue  with  Q-12)  X  No 

If  no,  please  specify  what  remains  to  be  done  and  when  you  anticipate  such  a  plan  to  be  completed  and 
ready  for  implementation. 

Plan  needed  to  be  written, 

Q-12.  Were  mere  any  other  Special  Team  activities  that  occurred  during  the  On-site  Visit  concerning 

development  of  a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes, 
please  explain. 


}l 
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Questionnaire  IB:  This  questionnaire  was  designed  to  assess  the  time  and  effort  of  pilot  plants  in  develop- 
ing the  plant-specific  plans  prior  to  and  during  the  On-Site  Visit. 


-1.  Who  on  your  staff  worked  on  the  adaptation  of  the  generic  HACCP  model  to  a  plant-specific  HACCP  plan? 
(titles /numbers  of  people  involved,  etc.) 

Quality  Control  Manager 

Vice-president  for  Operations 


'-2.  What  activities  were  undertaken  to  develop  the  plant-specific  plan  (e.g.,  meetings,  phone  calls,  training 
sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics  were  covered  and  what  was 
accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was  there  preparation  time  for  the 
activities?  If  so,  how  much  time? 

The  Quality  Control  Manager  and  Vice  President  for  Operations  met  with  the  HACCP  Special  Team 
to  decide  on  the  HACCP  product  for  this  pilot  test  The  Quality  Control  Manager  took  the  flip  chart 
sheets  used  at  the  awareness  training  and  spent  three  days  putting  together  the  plant-specific  plan 
based  on  the  generic  model.  The  plan  was  submitted  twice  to  FSIS  and  accepted  the  third  time. 

The  Quality  Control  Manager  said  determining  the  hazards  was  the  most  difficult  part  of  developing  the 
plant-specfic  plan. 

*-3.  Were  there  any  other  pilot  plant  activities  concerning  the  development  of  a  plant-specific  plan  based  on 
the  generic  model  that  occurred  prior  to  or  during  the  On-Site  Visit  that  we  haven't  discussed? 
If  yes,  please  explain. 


No. 
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Questionnaire  2A:  This  questionnaire  was  designed  for  the  Special  Team  members  at  the  completion  of 
the  plant-specific  plan.. 


Q-l.  Since  the  On-Site  Visit,  what  did  you  do  to  continue  the  development  of  the  plant-specific  HACCP  plan 
Please  be  specific  (meetings,  phone  calls,  etc;  how  many,  how  long,  with  whom). 

The  Special  Team  used  the  checklist  to  determine  if  the  plant-specific  plan  was  complete.  They  did 
this  in  stages,  slowly  getting  the  plan  to  the  point  where  it  could  be  approved.  Robert  Baele  made 
additional  visits  to  the  plant  to  help  management  make  changes  based  on  the  checklist  The  IIC  worli 
with  the  process  also.  The  Special  Team  thought  the  most  difficult  part  of  developing  the  plan  for  th 
plant  was  keeping  the  processes  separate. 

Additional  meetings  and  phone  conversations  were  held  with  plant  management.  The  plan  was  sent 
back  from  Was hing to n,D .  C .,  twice  for  revision  for  lack  of  specificity. 

Five  plant  visits  io  discuss  plant-specific  plan  11  hours 

62  phone  calls  from  August  to  March  7  V2  hours 


4, 


Q-2.  When  was  the  plant -specific  HACCP  plan  completed?  Tanuary  9, 1992_ 


1-3, 


Q-3.  Were  there  any  other  Special  Team  activities  that  occurred  after  the  On-Site  Visit  concerning 

development  of  a  plant  specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes, 
please  specify. 

One  Special  Team  member  though?;  there  were  too  many  people  participating  in  the  plant  walk  throw 
during  the  On-Site.  The  HACCP  Special  Team  members  assigned  to  this  plant  made  some  changes  ii 
the  plans  for  their  next  On-Site.  Based  on  this  experience,  their  next  On-Site  would  not  be  as  structui 
They  wouldn't  meet  separately  with  FSIS  inspectors  before  meeting  with  the  plant  management.  On 
Special  Team  member  felt  more  people  from  the  line  who  would  be  performing  the  monitoring 
functions  should  attend  HACCP  awareness  training  sessions, 
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Questionnaire  2B:  This  questionnaire  was  designed  for  plant  management  at  the  completion  of  the  plant- 
specific  plan. 


•1.  Since  the  On-Site  Visit,  what  activities  were  undertaken  to  develop  the  plant-specific  model 

(e.g.,  meetings,  phone  calls,  training  sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics 
were  covered  and  what  was  accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was 
there  preparation  time  for  the  activities?  If  so,  how  much  time? 

According  to  the  plant  Quality  Control  Manager,  no  changes  were  made  in  the  plant-specific  plan  since 
the  on-site.  (See  Questionnaire  2A  for  information  about  activities  after  the  On-Site.) 


-2.  Since  the  On-Site  Visit,  have  there  been  any  changes  in  the  numbers  or  types  staff  (titles)  who  worked  on 
the  development  of  a  plant-specific  HACCP  plan  based  on  the  generic  HACCP  model?  If  yes,  please 
specify. 

No. 


-3.  Were  there  any  other  pilot  plant  activities  that  occurred  after  the  On-Site  Visit  concerning  development  of 
a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes,  please  explain 

The  Quality  Control  Manager  said  he  had  had  prior  HACCP  experience.  He  found  it  difficult  to 
identify  the  hazards  and  CCPs  so  he  could  see  how  someone  unfamiliar  with  HACCP  would  find  it 
very  difficult 


APPENDIX  C 


RESPONSES  TO  COOKED  SAUSAGE 
QUESTIONNAIRE 


Cooked  Sausage  -  1 
Cooked  Sausage  -  2 
Cooked  Sausage  -  3 
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Questionnaire  1A:  This  questionnaire  was  designed  to  measure  Special  Team  activities  during  the  On- 
Site  Visit. 


PART  1:    Applicability  of  Generic  HACCP  Model 

Q-l.  What  activities  did  you  perform  to  determine  the  applicability  of  the  Generic  HACCP  Model  to  this  plant? 
Please  specify  activity  (e.g.,  used  checklists,  toured  plant,  met  with  various  people,  etc.)  and  the  amount  of 
time  expended. 

The  Special  Team  used  a  checklist  developed  by  the  HACCP  Special  Team  to  compare  plant 
processes  with  the  generic  model.  The  Special  Team  members  also  presented  an  HACCP  awareness 
session,  toured  the  plant,  and  met  with  plant  management  and  the  inspector-in-charge  (ICC). 

The  Special  Team  spent  three  days  with  plant  management  on  October  15-17, 1991. 

Q-2.  Based  on  the  On-Site  Visit,  was  the  generic  model  applicable  to  this  plant? 

 X  Yes,  Completely  Applicable   Yes,  Somewhat  Applicable   No,  Not  At  All  Applicable 

(Skip  to  PART  2)  (Continue  with  Q-3)  (Continue  with  Q-3) 

Q-3.  What  were  the  difficulties/problems  encountered?  Please  specify. 

No  difficulties. 

Q-4.  If  you  marked  Somewhat  or  No  in  Q-2,  whom  did  you  contact  about  the  difficulties /problems? 
(Check  all  that  apply.) 

HACCP  Special  Team  On-Site  Coordinator 

 HACCP  Special  Team  Director 

 Other  HACCP  Special  Team  MembeKs):  (name(s))   

 Other  FSIS  Personnel  (IIC,  Area,  Regional,  personnel,  etc.) 

 Plant  Personnel  (Management,  QC /Production,  etc.) 

 Other:  (title(s)/name(s))  _  — 

Q-5.  What  activities  were  undertaken  to  remedy  these  difficulties /problems  (meetings,  phone  calls,  etc.)? 
Please  specify  the  types  and  extent  of  contact  with  each  of  the  people/groups  marked  in  Q-4. 
(For  example:  an  estimate  such  as  "5  phone  calls  to  the  Special  Team  On-Site  Coordinator,  averaging 
15  minutes  each"  would  be  an  appropriate  description.) 

N/A 

Q-6.  To  address  the  difficulties  noted  in  Q-3,  did  the  pilot  plant  make  any  changes  to  their  processes/ 
procedures  to  enable  them  to  better  fit  the  Generic  HACCP  Model? 

 Yes   No  (Continue  with  Q-7) 

N/A 
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Q-7.  To  address  the  difficulties  noted  in  Q-3,  were  any  changes  made  to  the  Generic  HACCP  Model? 

N/A  Yes   No  (Continue  with  ( 


Q-8.  By  what  date  were  all  changes  incorporated  Into  the  generic  model?  N/A 
PART  2:     Plant-Specific  HACCP  Plan 


ft 


Q-9,  Did  the  pilot  plant  have  an  acceptable  plant-specific  plan  in  place  at  the  start  of  the  On-Site  Visit? 

_       Yes,  Completely  Acceptable    _____  Yes,  Partially  Acceptable    _NO  No  Plan/Unacceptable  P 
(Skip  to  Q-ll)  '  "  (Continue  with  Q-10)  (Continue  with  Q-10) 

Q-lG.If  you  marked  Partially  Acceptable  or  No/Unaccejdahle  in  Q-9,  what  did  you  do  to  develop  or  initiate 
development  of  a  plant-specific  HACCP  plan  (meetings /phone  calls,  etc.:  how  many,  how  long, 
with  whom,  etc.)? 

The  Special  Team  held  meetings  and  telephone  conversations  with  the  II C  and  plant  management 
They  worked  closely  with  plant  management  both  during  the  initial  On-Site  visit  and  at  a  follow»u| 
meeting  on  February  5-6, 1992, 

The  Special  Team  spent  two  days  each  on  follow-up  visits, 
Q-ll,  Was  a  plant-specific  HACCP  plan  completed  by  the  end  of  the  On-Site  Visit? 
.  _Yes  (Continue  with  Q-12)   X  No 

If  no,  please  specify  what  remains  to  be  done  and  when  you  anticipate  such  a  plan  to  be  completed  and 
ready  for  implementation. 


The  plant-specific  plan  was  completed  May  •%  1992. 


42.  Were  there  any  other  Special  Team  activities  that  occurred  during  the  On-site  Visit  concerning 
development  of  a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes, 
please  explain 

No, 
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Questionnaire  IB:  This  questionnaire  was  designed  to  assess  the  time  and  effort  of  pilot  plants  in  develop- 
ing plant-specific  plans  prior  to  and  during  the  On-Site  Visit. 


2-1.  Who  on  your  staff  worked  on  the  adaptation  of  the  generic  HACCP  model  to  a  plant-specific  HACCP  plan? 
(titles /numbers  of  people  involved,  etc.) 

Quality  Control  Manager  and  his  staff: 

Two  Microbiologist/Quality  Control  Technicians 

Two  in-plant  Quality  Control  Technicians. 


2-2.  What  activities  were  undertaken  to  develop  the  plant-specific  plan  (e.g.,  meetings,  phone  calls,  training 
sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics  were  covered  and  what  was 
accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was  there  preparation  time  for  the 
activities?  If  so,  how  much  time? 

Initially  all  the  quality-control  staff  sat  down  and  discussed  HACCP.  Then  the  quality  control  manager, 
and  the  two  Micro/QC  people  developed  the  plant-specific  plan.  They  had  to  work  after  hours  and  at 
night  to  use  their  one  computer  which  was  running  formulas  all  day.  To  establish  time/temperature 
limits,  they  looked  at  ten  previous  production  sheets  for  each  reading.  They  had  problems  listing  on 
the  plan  the  employee  who  was  responsible  for  verification  and  evaluation  tasks  at  some  CCPs. 

Write  plan  =  180  hrs. 

Changes  and  typing  =  55  hrs.  (One  person  said  she  had  used  23  hrs  of  overtime  to  type  up  the  HACCP 
plan.) 

Phone  calls  =11/2  hrs. 

Q-3.  Were  there  any  other  pilot  plant  activities  concerning  the  development  of  a  plant-specific  plan  based  on 
the  generic  model  that  occurred  prior  to  or  during  the  On-Site  Visit  that  we  haven't  discussed? 
If  yes,  please  explain. 


No. 
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Questionnaire  2A:  This  questionnaire  was  designed  for  the  Special  Team  members  at  the  completion 
the  plant -specific  plan. 


Q-l.  Since  the  On-Site  Visit,  what  did  you  do  to  continue  the  development  of  the  plant-specific  HACCP  plan 
Please  be  specific  (meetings,  phone  calls,  etc.:  how  many,  how  long,  with  whom). 

Combination  of  meetings  with  the  IIC  and  plant  management  and  telephone  calls  to  same. 

Approximate  time  =  16  hours 

Approximate  time  =  10  calls  ®  20  minutes  each  ■  200  minutes 
Q-2.  When  was  the  plant  .specific  HACCP  plan  completed?        ...  ..May  4, 1992 


Q -3.  Were  there  any  other  Special  Team  activities  that  occurred  after  the  On-Site  Visit  concerning 

development  of  a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes, 
please  specify. 

No. 
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Questionnaire  2B:  This  questionnaire  was  designed  for  plant  managment  at  the  completion  of  the  plant- 
specific  plan. 


J-l.  Since  the  On-Site  Visit,  what  activities  were  undertaken  to  develop  the  plant-specific  model 

{e.g.,  meetings,  phone  calls,  training  sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics 
were  covered  and  what  was  accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was 
there  preparation  time  for  the  activities?  If  so,  how  much  time? 

The  plan  was  sent  back  for  revisions.  More  specificity  was  needed  in  the  plan.  In  particular,  the  plant 
needed  to  designate  which  employee  did  which  task  at  each  CCP.  The  plan  was  sent  back  to  FSIS  with 
the  changes  encorporated  in  it  It  was  returned  again  for  a  few  minor  changes. 

Changes  to  plan  and  forms  updating  =  25  hrs. 


Since  the  On-Site  Visit,  have  there  been  any  changes  in  the  numbers  or  types  staff  (titles)  who  worked  on 
the  development  of  a  plant-specific  HACCP  plan  based  on  the  generic  HACCP  model?  If  yes,  please 
specify. 

No. 


2-3.  Were  there  any  other  pilot  plant  activities  that  occurred  after  the  On-Site  Visit  concerning  development  of 
a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes,  please  explain. 

The  Quality  Control  Manager  had  attended  the  HACCP  workshop.  The  assistant  had  some  food  science 
training.  The  other  quality  control  staff  were  lab  technicians. 
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Questionnaire  1A:  This  questionnaire  was  designed  to  measure  Special  Team  activities  during  the  On- 
Site  Visit.  6 


PART  1:     Applicability  of  Generic  HACCP  Model 

Q-L  What  activities  did  you  perform  to  determine  the  applicability  of  the  Generic  HACCP  Model  to  this  plant? 
Please  specify  activity  (e.g.,  used  checklists,  toured  plant,  met  with  various  people,  etc.)  and  the  amount  of 
time  expended. 

The  date  of  the  on-site  was  January  21-23, 1992. 

The  Special  Team  toured  the  plant  to  see  if  the  process  was  applicable  to  the  generic  model,  looked 
at  the  layout  of  the  plant  and  the  other  products  produced,  met  with  plant  management 

Q-2.  Based  on  the  On-Site  Visit,  was  the  generic  model  applicable  to  this  plant? 

__X_  Yes,  Completely  Applicable   Yes,  Somewhat  Applicable   No,  Not  At  All  Applicable 

(Skip  to  PART  2)  (Continue  with  Q-3)  (Continue  with  Q-3) 

Q-3.  What  were  the  difficulties /problems  encountered?  Please  specify. 

No  difficulties. 

Q-4.  If  you  marked  Somewhat  or  No  in  Q-2,  whom  did  you  contact  about  the  difficulties /problems? 
(Check  all  that  apply.) 

HACCP  Special  Team  On-Site  Coordinator 

HACCP  Special  Team  Director 

Other  HACCP  Special  Team  MembeKs):  (name(s))  


_Other  FSIS  Personnel  (IIC,  Area,  Regional,  personnel,  etc.) 
_Plant  Personnel  (Management,  QC /Production,  etc.) 
.Other:  (title(s)/name(s))  


Q-5.  What  activities  were  undertaken  to  remedy  these  difficulties /problems  (meetings,  phone  calls,  etc.)? 
Please  specify  the  types  and  extent  of  contact  with  each  of  the  people /groups  marked  in  Q-4. 

N/A 

Q-6.  To  address  the  difficulties  noted  in  Q-3,  did  the  pilot  plant  make  any  changes  to  their  processes/ 
procedures  to  enable  them  to  better  fit  the  Generic  HACCP  Model? 

N/A   Yes   No  (Continue  with  Q-7) 

Q-7.  To  address  the  difficulties  noted  in  Q-3,  were  any  changes  made  to  the  Generic  HACCP  Model? 

N/A   Yes   No  (Continue  with  Q-8) 

Q-8.  By  what  date  were  all  changes  incorporated  into  the  generic  model?  N/A  
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PART  2x     Plant-Specific  HACCP  Plan 

Q-9.  Did  the  pilot  plant  have  an  acceptable  plant-specific  plan  in  place  at  the  start  of  the  On-Site  Visit? 

Yes,  Completely  Acceptable  Yes,  Partially  Acceptable   No  Plan /Unacceptable1 

(Skip  to  Q-11)  (Continue  with  Q-1.0)  (Continue  with  Q-10) 

Q-lO.If  you  marked  Partially  Acceptable  or  No/Unacceptable  in  Q-9,  what  did  you  do  to  develop  or  initial 
development  of  a  plant-specific  HACCP  plan  (meetings/phone  calls,  etc.:  how  many,  how  long, 
with  whom,  etc.)? 

Special  Team  members  talked  on  die  telephone  with  plant  personnel,  stopped  to  visit  the  plant,  an 
worked  incrementally  to  bring  the  plant-specific  plan  to  an  acceptable  level. 


Q-11.  Was  a  plant- specific  HACCP  plan  completed  by  the  end  of  the  On-Site  Visit? 

__  _Yes  (Continue  with  Q-l  2)  _X  No 

If  no,  please  specify  what  remains  to  be  done  and  when  do  you  anticipate  such  a  pain  to  be  completed 
ready  for  implementation 

Q-12.  Were  there  any  other  Special  Team  activities  that  occurred  during  the  On-site  Visit  concerning 

development  of  a  plant-specific  plan  basM  on  the  generic  model  that  we  haven't  discussed?  If  yes, 
please  explain. 

The  plant  is  a  small  (50  employees)  one.  Th&  Quality  Control  Manager  had  been  there  12  years. 
It  was  necessary  to  start  from  ground  zero  getting  them  up  to  speed  on  HACCP  They  had  just  start 

a  TQC  program. 

'It  was  important  to  include  the  date  collector  and  back-up,  the  regional  coordinator,  the  Processing 
Inspection  Coordinator  (PIC),  and  other  inspectors  in  at  the  on-site  so  that  all  could  move  forward 
together.  "The  Special  Team  member  strongly  recommended  that  the  data  collector  be  chosen  before 

any  on-site  visit  is  done  in  the  future. 
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Questionnaire  IB:  This  questionnaire  was  designed  to  assess  the  time  and  effort  of  pilot  plants  in  develp- 
ing  the  plant-specific  plans  prior  to  and  during  the  On-Site  Visit. 


3-1.  Who  on  your  staff  worked  on  the  adaptation  of  the  generic  HACCP  model  to  a  plant-specific  HACCP  plan? 
(titles /numbers  of  people  involved,  etc.) 


Quality  Control  Manager. 


What  activities  were  undertaken  to  develop  the  plant-specific  plan  (e.g.,  meetings,  phone  calls,  training 
sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics  were  covered  and  what  was 
accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was  there  preparation  time  for  the 
activities?  If  so,  how  much  time? 


See  Questionnaire  2B. 


Q-3.  Were  there  any  other  pilot  plant  activities  concerning  the  development  of  a  plant-specific  plan  based  on 
the  generic  model  that  occurred  prior  to  or  during  the  On-Site  Visit  that  we  haven't  discussed? 
If  yes,  please  explain. 


No. 
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Questionnaire  2A;  This  questionnaire  was  designed  for  the  Special  Team  members  at  the  completion 
the  plant-specific  plan. 


Since  the  On-Site  Visit,  what  did  you  do  to  continue  the  development  of  the  plant-specific  HACCP  plar 
Please  be  specific  (meetings,  phone  calls,  etc.:  how  many,  how  long,  with  whom). 


The  original  Quality  Control  Manager  was  fired  and  a  new  manager  was  hired  during  the  time  the  p 
specific  plan  was  being  developed..  The  implementation  phase  of  the  study  began  before  the  final  p 
specific  plan  was  approved. 


Q-l,  When  was  tho  plant-specific  HACCP  plan  completed?.,  January  1993. 


Q-3.  Were  there  any  other  Special  Team  activities  that  occurred  after  the  On-Site  Visit  concerning 

develoment  of  a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes, 
please  specify  . 

No. 
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Questionnaire  2B:  This  questionnaire  was  designed  for  plant  management  at  the  completion  of  the  plant- 
specific  plan. 


Since  the  On-Site  Visit,  what  activities  were  undertaken  to  develop  the  plant-specific  model 
(e.g.,  meetings,  phone  calls,  training  sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics 
were  covered  and  what  was  accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was 
there  preparation  time  for  the  activities?  If  so>  how  much  time? 


The  new  Quality  Control  Manager  estimated  he  had  spent  approximately  30  hours  developing  the 
plant-specific  plan.  He  said  he  had  probably  spent  another  30  hours  learning  about  HACCP  since 
he  had  not  been  there  for  the  On-Site  visit 


)-2.  Since  the  On-Site  Visit,  have  there  been  any  changes  in  the  numbers  or  types  staff  (titles)  who  worked  on 
the  development  of  a  plant-specific  HACCP  plan  based  on  the  generic  HACCP  model?  If  yes,  please 
specify. 

A  new  Quality  Control  Manager  was  hired. 


>-3.  Were  there  any  other  pilot  plant  activities  that  occurred  after  the  On-Site  Visit  concerning  development  of 
a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes,  please  explain. 

No. 
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Questionnaire  1A:  This  questionnaire  was  designed  to  measure  Special  Team  activities  durinv  the  On- 
Site  Visit. 


PART  1:    Applicability  of  Generic  HACCP  Model 

Q-l.  What  activities  did  you  perform  to  determine  the  applicability  of  the  Generic  HACCP  Model  to  this  plant? 
Please  specify  activity  (e.g.,  used  checklists,  toured  plant,  met  with  various  people,  etc.)  and  the  amount  of 
time  expended. 

Because  the  pilot  plant  was  small  and  plant  management  felt  that  they  couldn't  have  all  the  employees 
they  wanted  to  attend  the  session  out  of  the  plant  at  one  time,  the  On-site  at  CS-3  was  held  at  two 
sessions,  April  22-23  and  April  27-29, 1992.  At  the  first  session,  the  plant  manager,  Quality  Control 
Manager,  and  maintenance  manager  attended  from  the  plant  The  PIC,  Performance  Based  Inspection 
System  (PBIS)  management  analyst;  back-up  inspector ,  the  data  collector,  and  area  representative 
attended.  The  second  session  included  the  owner,  the  sales  manager  and  the  laboratory  person  for  the 
plant  FSIS  representatives  included  the  inspector-in-charge,  the  backup  inspector,  circuit  supervisor, 
and  the  backup  data  collector. 

Special  Team  members  walked  through  the  flow  diagram.  Each  plant  group  had  a  different  perception 
and  their  flow  diagrams  were  different  They  couldn't  believe  it,  but  got  together  and  learned  where 
they  differed.  The  Special  Team  members  thought  gaining  the  perspective  of  the  sales  manager  was 
useful  in  a  total  TQM  program. 

Q-2.  Based  on  the  On-Site  Visit,  was  the  generic  model  applicable  to  this  plant? 

__X_  Yes,  Completely  Applicable   Yes,  Somewhat  Applicable   No,  Not  At  All  Applicable 

(Skip  to  PART  2)  (Continue  with  Q-3)  (Continue  with  Q-3) 

Q-3.  What  were  the  difficulties/problems  encountered?  Please  specify. 

The  only  difference  between  the  generic  model  and  the  plant-specific  plan  was  that  the  plant  uses 
natural  casings  and  adds  nitrite  at  two  different  levels. 


Q-4.  If  you  marked  Somewhat  or  No  in  Q-2,  whom  did  you  contact  about  the  difficulties /problems? 
(Check  all  that  apply.) 

 HACCP  Special  Team  On-Site  Coordinator 

HACCP  Special  Team  Director 

Other  HACCP  Special  Team  Member(s):  (name(s))   

Other  FSIS  Personnel  (IIC,  Area,  Regional,  personnel,  etc.) 

 Plant  Personnel  (Management,  QC /Production,  etc.) 

Other:  (title(s)/name(s))  .  

Q-5.  What  activities  were  undertaken  to  remedy  these  difficulties/problems  (meetings,  phone  calls,  etc.)? 
Please  specify  the  types  and  extent  of  contact  with  each  of  the  people/groups  marked  in  Q-4. 
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N/A 

Q-6.  To  address  the  difficulties  noted  in  Q-3,  did  ihe  pilot  plant  make  any  changes  to  their  processes/ 
procedures  to  enable  them  to  better  fit  the  Generic  HACCP  Model? 

N/A  Yes   No  (Continue  with  I 


Q-7.  To  address  the  difficulties  noted  in  Q  3,  were  any  changes  made  to  the  Generic  HACCP  Model? 

N/A  ,   Yes    _No  (Continue  with  ( 


Q-8.  By  what  date  were  all  changes  incorporated  into  the  generic  model?_  N/A„ 

PART  2:     Plant-Specific  HACCP  Plan 


It 


Q-9.  Did  the  pilot  plant  have  an  acceptable  plant  specific  plan  in  place  at  the  start  of  the  On-Site  Visit? 

Yes,  Completely  Acceptable     _____  Yes,  Partially  Acceptable    _X  No  Plan/Unacceptable  Pie' 

(Skip  to  Q-ll)  (Continue  with  Q-10)  (Continue  with  Q-10) 

Q-10.  If  you  marked  Partially  Acceptable  or  No/Unacceptable  in  Q-9,  what  did  you  do  to  develop  or  initiate 
development  of  a  plant-specific  HACCP  plan  (meetings/phone  calls,  etc.:  how  many,  how  long, 
with  whom,  etc )? 


Q-ll,  Was  a  plant-specific  HACCP  plan  completed  by  the  end  of  the  On-Site  Visit? 

 Yes  (Continue  with  Q-12)  __X_No 

If  no,  please  specify  what  remains  to  be  done  and  when  you  anticipate  such  a  plan  to  be  completed  an( 
ready  for  implementation. 


Q-3, 


Q-12.  Were  there  any  other  Special  Team  activities  that  occurred  during  the  On-site  Visit  concerning 

development  of  a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes, 
please  explain. 

The  plant  group  which  met  the  second  session  misunderstood  what  the  HACCP  plan  was.  When  the 
met  with  the  first  group  who  were  the  operations  people,  they  were  better  able  to  understand  the  pro 
The  Regional  HACCP  Coordinator,  met  with  them  one  or  two  times  afterwards  to  help  them  work  or 
their  HACCP  plan. 
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Questionnaire  IB:  This  questionnaire  was  designed  to  assess  the  time  and  effort  of  pilot  plants  in  develop- 
ing the  plant-specific  plans  prior  to  and  during  the  On-Site  Visit. 


2-1.  Who  on  your  staff  worked  on  the  adaptation  of  the  generic  HACCP  model  to  a  plant-specific  HACCP  plan? 
(titles /numbers  of  people  involved,  etc.) 


wiJ 


Quality  Control  Manager 


an 


2-2.  What  activities  were  undertaken  to  develop  the  plant-specific  plan  {e.g.,  meetings,  phone  calls,  training 
sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics  were  covered  and  what  was 
accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was  there  preparation  time  for  the 
activities?  If  so,  how  much  time? 


It  took  15  hours  for  the  Quality  Control  Manager  to  put  the  plant-specific  plan  on  paper. 

Five  hours  were  spent  in  meetings  with  production  people  to  verify  CCPs,  controls,  etc. 

The  hand-written  plant-specific  plan  was  given  to  the  FSIS  Regional  Office  to  put  into  the  format  of 
the  generic  model. 


2-3.  Were  there  any  other  pilot  plant  activities  concerning  the  development  of  a  plant-specific  plan  based  on 
the  generic  model  that  occurred  prior  to  or  during  the  On-Site  Visit  that  we  haven't  discussed? 
If  yes,  please  explain. 

No. 


ntl 
epi 
fk 
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Questionnaire  2A:  This  questionnaire  was  designed  for  the  Special  Team  members  at  the  completion 
the  plant-specific  plan. 


Q-l.  Since  the  On  Site  Visit,  what  did  you  do  to  continue  the  development  of  the  plant-specific  HACCP  pla 
Please  be  specific  (meetings,  phone  calls,  etc.:  how  many,  how  Jong,  with  whom). 


Eighteen  hours  with  plant  personnel  August  18-19, 1992, 

Phone  calls  (including  conference  calls)  approximately  four  and  one  half  hours  total  time.  All  calls 
were  with  plant  HACCP  Coordinator  and  other  plant  personnel 


Q-2.  When  was  the  plant  specific  HACCP  plan  completed?   November  2, 1992  


Q-3.  Were  there  any  other  Special  Team  activities  that  occurred  after  tine  On~Site  Visit  concerning 

development  of  a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes, 
please  specify, 


The  office  of  the  Regional  HACCP  Coordinator  put  their  plant-specific  plant  on  the  computer  and 
talked  with  plant  officials.  Three  versions  were  typed  into  the  computer  before  the  final  version 
was  accepted  for  use.  The  regional  HACCP  coordinator  estimated  that  his  staff  ?  pent  eight  to  ten 
hours  typing  the  plan  and  alterations  to  it 
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Questionnaire  2B:  This  questionnaire  was  designed  for  plant  management  at  the  completion  of  the  plant- 
specific  plan. 


Q-L  Since  the  On-Site  Visit,  what  activities  were  undertaken  to  develop  the  plant-specific  model 

(e.g.,  meetings,  phone  calls,  training  sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics 
were  covered  and  what  was  accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was 
there  preparation  time  for  the  activities?  If  so,  how  much  time? 


See  Questionnaire  IB. 


Q-2.  Since  the  On-Site  Visit,  have  there  been  any  changes  in  the  numbers  or  types  staff  (titles)  who  worked  on 
the  development  of  a  plant-specific  HACCP  plan  based  on  the  generic  HACCP  model?  If  yes,  please 
specify. 


After  the  Quality  Control  Manager  put  together  the  plant-specific  plan,  she  resigned.  Further  changes 
in  the  plan  were  made  by  the  plant  owner. 


Q-3.  Were  there  any  other  pilot  plant  activities  that  occurred  after  the  On-Site  Visit  concerning  development  of 
a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes,  please  explain. 


No. 
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Questionnaire  1A:  This  questionnaire  was  designed  to  measure  Special  Team  activities  durins  the  On- 
Site  Visit  ^  6 


PART  1:    Applicability  of  Generic  HACCP  Model 

Q-l.  What  activities  did  you  perform  to  determine  the  applicability  of  the  Generic  HACCP  Model  to  this  plant? 
Please  specify  activity  (e.g.,  used  checklists,  toured  plant,  met  with  various  people,  etc.)  and  the  amount  of 
time  expended. 

About  35  persons  attended  the  On-site  training  at  this  NELS  plant  which  was  held  in  a  trailer  conference 
room  at  the  plant  on  May  26-28, 1992.  Attending  were  four  VMOs,  two  GS-8s,  two  data  collectors,  and 
some  area  inspectors,  and  various  FSIS  people  from  Washington ;  eight  to  ten  line  and  quality  control 
plant  people,  the  area  HACCP  coordinator,  the  area  Quality  Control  officer,  and  an  area  poultry  staff 
officer.  The  Special  Team  held  an  awareness  session,  toured  the  plant  (in  two  groups,  mostly  FSIS 
staff),  and  taught  the  participants  how  to  identify  CCPs  and  develop  a  HACCP  plan. 

Special  Team  members  held  two  HACCP  information  sharing  sessions  with  the  line  inspectors,  both 
-    day  and  night  shifts.  About  twenty  people  were  there  altogether.  Inspectors  were  given  one  and  one 
half  hour  of  administrative  overtime,  but  stayed  about  two  hours.  The  Special  Team  members  planned 
to  meet  with  the  line  inspectors  again  at  the  training  session.  The  inspectors  posed  good  questions. 

Q-2.  Based  on  the  On-Site  Visit,  was  the  generic  model  applicable  to  this  plant? 

 X  Yes,  Completely  Applicable   Yes,  Somewhat  Applicable   No,  Not  At  All  Applicable 

(Skip  to  PART  2)  (Continue  with  Q-3)  (Continue  with  Q-3) 

Q-3.  What  were  the  difficulties/problems  encountered?  Please  specify. 

No  difficulties. 

Q-4.  If  you  marked  Somewhat  or  No  in  Q-2,  whom  did  you  contact  about  the  difficulties/problems? 
(Check  all  that  apply.) 

HACCP  Special  Team  On-Site  Coordinator 

HACCP  Special  Team  Director 

Other  HACCP  Special  Team  Members):  (name(s))  


.Other  FSIS  Personnel  (EC,  Area,  Regional,  personnel,  etc.) 
_Plant  Personnel  (Management,  QC/ Production,  etc.) 
Other:  (title(s)/name(s))  


Q-5.  What  activities  were  undertaken  to  remedy  these  difficulties /problems  (meetings,  phone  calls,  etc.)? 
Please  specify  the  types  and  extent  of  contact  with  each  of  the  people/groups  marked  in  Q-4. 

There  was  a  good  fit  between  the  generic  model  and  the  plant  slaughter  process.  Plant  management  was 
hesitant  at  first  to  add  any  additional  steps  that  were  not  in  the  generic  model  for  fear  that  these 
additions  would  become  regulations.  However,  their  plant  specific-plan  included,  at  least  in  the  flow 
diagram,  steps  like  the  chlorine  rinse  they  include  before  final  packaging,  the  post  scald  rinse,  and 
chlorine  at  other  points  on  the  line. 


Poultry  Slaughter 


Q-6.  To  address  the  difficulties  noted  in  Q-3,  did  the  pilot  plant  make  any  changes  to  their  processes/ 
procedures  to  enable  them  to  better  fit  the  Generic  HACCP  Model? 

 Yes  No  (Continue  with  C 

N/A 

Q-7.  To  address  the  difficulties  noted  in  Q-3,  were  any  changes  made  to  the  Generic  HACCP  Model? 

N/A   Yes  No  (Continue  with  Q 

Q-8.  By  what  date  were  all  changes,  incorporated  Into  the  generic  modei?_  N/A  

PAFT  2:     Plant-Specific  HACCP  Plan 

Q-9.  Did  the  pilot  plant  have  an  acceptable  plant-specific  plan  in  place  at  the  start  of  the  On-Site  Visit? 

Yes,  Completely  Acceptable    ____  Yes,  Partially  Acceptable      X  No  Plan/Unacceptable  Pla 

(SkiptoQ-11)  (Continue  with  Q-10)  (Continue  with  Q-10) 

Q-lO.If  you  marked  Partially  Acceptable  or  No/Unacceptable  in  Q-9,  what  did  you  do  to  develop  or  initiate 
development  of  a  plant-specific  HACCP  plan  (meetings/phone  calls,  etc:  how  many,  how  long, 
with  whom,  etc.)? 

... 

The  plant  had  no  plant-specific  plan  at  the  start  ©f  the  On-site.  No  one  from  the  plant  had  attended 
the  Poultry  Slaughter  workshop  so  plant  staff  had  no  idea  how  to  identify  CCPs  and  what  the  next 
steps  would  be  to  design  a  plant-specific  plan, 

Q  41,  Was  a  plant-specific  HACCP  plan  completed  by  the  end  of  the  On-Site  Visit? 

Yes  (Continue  with  Q-12)  .  X  No 

If  no,  please  specify  what  remains  to  be  done  and  when  you  anticipate  such  a  plan  to  be  completed  and 
ready  for  implementation. 

Special  Team  members  visited  at  the  plant:  on  Jmtf  14-15, 1992,  to  help  them  complete  their  plant 
specific  plan, 

All  four  lines  were  a  part  of  the  HACCP  pilot  Although  two  lines  could  be  isolated  as  going  into 
one  chiller,  all  four  lines  dump  into  a  common  area  so  two  lines  couldn't  be  isolated  for  the  pilot  H 
Special  Team  also  extended  the  plant-specific  plan  through  to  the  cooler  because  of  the  layout  of  the 
plant 

Q-12.  Were  mere  any  other  Special  Team  activities  that  occurred  during  the  On-site  Visit  concerning 

development  of  a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes, 
please  explain . 

The  plant  had  a  good  person  as  HACCP  coordinator  for  the  plant  The  plant  management 
was  receptive  and  interested.  Although  no  one  from  the  plant  attended  the  HACCP  Poultry  Slaughtt 
workshop,  a  retired  company  employee  who  did  attend  encouraged  the  plant  to  participate  as  a  pilot 
and  has  been  acting  as  a  consultant  Plant  management  felt  that  if  HACCP  goes  nationwide, 
they  would  like  to  be  in  the  forefront 

Initially  there  had  been  some  problem  with  the  data  collectors  and  sample  collection  at  the  plant. 
A  staff  person  from  Washington,!?  C,  went  to  the  plant  to  straighten  things  out 


1 
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Questionnaire  IB:  This  questionnaire  was  designed  to  assess  the  time  and  effort  of  pilot  plants  in  develop- 
ing plant-specific  plans  prior  to  and  during  the  On-Site  Visit. 


2-1.  Who  on  your  staff  worked  on  the  adaptation  of  the  generic  HACCP  model  to  a  plant-specific  HACCP  plan? 
(titles /numbers  of  people  involved,  etc.) 

The  HACCP  Coordinator  adopted  the  generic  model  to  the  plant-specific  plan  with  input  from  the  2nd 
Processing  Plant  Manager,  the  1st  Processing  Plant  Manager,  the  Quality  Assurance  Manager,  the 
Wastewater  Manager,  the  2nd  Processing  Superintendent,  and  other  production  foremen  throughout 
the  plant 

ablefi  Q-2.  What  activities  were  undertaken  to  develop  the  plant-specific  plan  (e.g.,  meetings,  phone  calls,  training 
sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics  were  covered  and  what  was 
accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was  there  preparation  time  for  the 
activities?  If  so,  how  much  time? 

Conference  call  with  Carlson  and  Clark  to  revise  plant 

specific  HACCP  plan  July  7, 1992.  45min. 
June  23, 1992,  Meeting  with  plant  management  prior  to  on-site  visit 

to  prepare  for  on-site  visit  2  hrs. 

Developing  plant  specific  plant  46  hrs. 

Working  with  Data  Collectors  -  assisting  first  three  days  2  hrs. 

Interview  with  Dr.  Henderson,  Dr.  Arlington,  and  John  Snyder  1  nr. 
Second  draft  of  HACCP  plan  with  Carson  and  Clark  review  with 

plant  management  20  hrs. 

Meeting  with  Department  foreman  to  update  on  HACCP  1  nr. 

Q-3.  Were  there  any  other  pilot  plant  activities  concerning  the  development  of  a  plant-specific  plan  based  on 
the  generic  model  that  occurred  prior  to  or  during  the  On-Site  Visit  that  we  haven't  discussed? 
If  yes,  please  explain. 

No. 
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Questionnaire  2A:  This  questionnaire  was  designed  for  the  Special  Team  members  at  the  completion  oj 
the  plant-specific  plan. 


Q-L  Since  the  On-Site  Visit,  what  did  you  do  to  continue  the  development  of  the  plant-specific  HACCP  plan 
Please  be  specific  (meetings,  phone  calls,  etc.:  how  many,  how  long,  with  whom). 


Sixteen  hours  with  plant  HACCP  Coordinator  July  14-15, 1992,  at  the  plant 

Phone  calls  (including  conference  calls)  approximately  six  hours  total  time.  All  calls  were  with  plant 
HACCP  Coordinator  and  other  plant  personnel  (e  g,  plant  manager.) 


Q-2,  When  was  the  plant-specific  HACCP  plan  completed?       .  „„_July  30, 1992. 


Q-3.  Were  there  any  other  Special  Team  activities  that  occurred  after  the  On-Site  Visit  concerning 

development  of  a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes, 
please  specify. 


No, 
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Questionnaire  2B:  This  questionnaire  was  designed  for  plant  management  at  the  completion  of  the  plant- 
specific  plan. 


1.  Since  the  On-Site  Visit,  what  activities  were  undertaken  to  develop  the  plant-specific  model 

(e.g.,  meetings,  phone  calls,  training  sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics 
were  covered  and  what  was  accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was 
there  preparation  time  for  the  activities?  If  so,  how  much  time? 


See  Questionnaire  IB  Question  Q-2. 


Since  the  On-Site  Visit,  have  there  been  any  changes  in  the  numbers  or  types  staff  (titles)  who  worked  on 
the  development  of  a  plant-specific  HACCP  plan  based  on  the  generic  HACCP  model?  If  yes,  please 
specify. 


No. 

3.  Were  there  any  other  pilot  plant  activities  that  occurred  after  the  On-Site  Visit  concerning  development  of 
a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes,  please  explain. 

No. 
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Questionnaire  1A:  This  questionnaire  was  designed  to  measure  Special  Team  activities  during  the  On- 
Site  Visit 


PART  1:    Applicability  of  Generic  HACCP  Model 

Q-l.  What  activities  did  you  perform  to  determine  the  applicability  of  the  Generic  HACCP  Model  to  this  plant? 
Please  specify  activity  (e.g.,  used  checklists,  toured  plant,  met  with  various  people,  etc.)  and  the  amount  of 
time  expended. 

The  On-Site  visit  at  this  plant  was  held  May  27-28, 1992.  The  Special  Team  used  the  HACCP  checklists, 
toured  the  plant  and  gave  HACCP  familiarization  sessions.  There  were  about  25  people  at  the  On-Site, 
half  FSIS  staff  and  half  plant  personnel  The  plant  had  a  draft  HACCP  plan  prepared  prior  to  the 
On-Site.  They  had  had  help  developing  it  from  the  quality  control  staff  of  a  poultry  slaughter  plant 
owned  by  their  company  which  is  a  HACCP  pilot  plant 

The  Special  Team  held  information  sessions  about  HACCP  for  FSIS  line  inspectors:  one  session  for  each 
shift,  15  attended  one  session  and  eight  or  nine  the  other.  Inspectors  were  paid  administrative  overtime 
for  the  sessions.  The  sessions  were  so  successful  that  the  team  planned  to  hold  special  sessions  for  line 
inspectors  again  during  training. 

Q-2.  Based  on  the  On-Site  Visit,  was  the  generic  model  applicable  to  this  plant? 

__X_  Yes,  Completely  Applicable   Yes,  Somewhat  Applicable   No,  Not  At  All  Applicable 

(Skip  to  PART  2)  (Continue  with  Q-3)  (Continue  with  Q-3) 

Q-3.  What  were  the  difficulties/problems  encountered?  Please  specify. 

The  plant-specific  plan  did  not  contain  all  the  steps  that  the  generic  model  contains.  It  did  not  include 
packaging  and  storage.  This  was  not  a  problem  as  this  particular  plant  did  not  have  these  steps  as  part 
of  its  operations. 

Q-4.  If  you  marked  Somewhat  or  No  in  Q-2,  whom  did  you  contact  about  the  difficulties /problems? 
(Check  all  that  apply.) 

!  HACCP  Special  Team  On-Site  Coordinator 

 HACCP  Special  Team  Director 

Other  HACCP  Special  Team  Member(s):  (name(s))  .  

 Other  FSIS  Personnel  (IIC,  Area,  Regional,  personnel,  etc.) 

 Plant  Personnel  (Management,  QC /Production,  etc.) 

 Other:  (title(s)/name(s))_  - —  

Q-5.  What  activities  were  undertaken  to  remedy  these  difficulties/problems  (meetings,  phone  calls,  etc.)? 
Please  specify  the  types  and  extent  of  contact  with  each  of  the  people/groups  marked  m  Q-4. 


N/A 
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Q-6.  To  address  the  difficulties  noted  in  Q-3,  did  the  pilot  plant  make  any  changes  to  their  processes/ 
procedures  to  enable  them  to  better  £i t  the  Generic  HACCP  Model  ? 

 Yes  No  (Continue  with  Q 

N/A 

Q-7.  To  address  the  difficulties  noted  in  Q-3,  were  any  changes  made  to  the  Generic  HACCP  Model? 

Isjf/A  ™_  Yes    .  No  (Continue  with  Q 


Q-8.  By  what  date  were  aH  changes  incorporated  into  the  generic  mode! ?.  _  N/A_ 

PART  2:     Plant-Specific  HACCP  Plan 


1 


Q-9.  Did  the  pilot  plant  have  an  acceptable  plant-specific  plan  in  place  at  the  3*art  of  the  On-Site  Visit? 

Yes,  Completely  Acceptable  _  „_  Yes,  Partially  Acceptable  „X_No  Plan/Unacceptable  Pla! 
(SkiptoQ-U)  (Continue  with  Q-10)  (Continue  with  Q-10) 

Q-lO.If  you  marked  Partially  Acceptable  or  No/Unacceptable  in  Q-9,  what  did  you  do  to  develop  or  initiate  i 
development  of  a  plant-specific  HACCP  plan  (meetings/phone  calls,  etc :  how  many,  how  long, 
with  whom,  etc.)? 

The  plant  had  a  draft  plan  ready  for  review.  The  Quality  Control  Manager  at  another  pilot  plant  wo 
with  this  plant's  quality  control  staff.  All  the  plant's  operations  were  under  the  HACCP  plan  as  the 
plant  has  three  lines,  but  only  two  chill  tanks. 


(Ml. Was  a  plant-specific  HACCP  plan  completed  by  the  end  of  the  On-Site  Visit? 

__Yes  (Continue  with  Q-12)  .  ,_X  No 

If  no,  please  specify  what  remains  to  be  done  and  when  you  anticipate  such  a  plan  to  be  completed  anc 
ready  for  implementation. 


8 


•12. Were  there  any  other  Special  Team  activities  that  occurred  during  the  On-site  Visit  concerning 
development  of  a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes, 
please  explain 

The  plant  has  a  good  quality  control  group  arid  h  extremely  cooperative.  During  baseline  data 
collection,  chiorination,  already  in  use  in  this  plant?  was  discontinued.  It  was  added  during  imple- 
mentation as  part  of  their  HACCP  plan. 
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Questionnaire  IB:  This  questionnaire  was  designed  to  assess  the  time  and  effort  of  pilot  plants  in  develop- 
ing plant-specific  plans  prior  to  and  during  the  On-Site  Visit. 


Who  on  your  staff  worked  on  the  adaptation  of  the  generic  HACCP  model  to  a  plant-specific  HACCP  plan? 
(titles /numbers  of  people  involved,  etc.) 

Quality  Control  Manager 

Plant  Manager 

Quality  Control  Manager  (of  other  company  owned  plant) 


>-2.  What  activities  were  undertaken  to  develop  the  plant-specific  plan  (e.g.,  meetings,  phone  calls,  training 
sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics  were  covered  and  what  was 
accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was  there  preparation  time  for  the 
activities?  If  so,  how  much  time? 

Three  meetings  were  held  to  develop  and  revise  plant  specific  plan  12  hrs. 
One  meeting  was  held  to  assign  duties  of  plant  specific  plan  2  hrs. 

One  meeting  was  held  to  explain/get  comments  from  supervisors  2  hrs. 

Phone  calls  to  review  and  consulting  for  plant  specific  plan  3  hrs. 


>-3.  Were  there  any  other  pilot  plant  activities  concerning  the  development  of  a  plant-specific  plan  based  on 
the  generic  model  that  occurred  prior  to  or  during  the  On-Site  Visit  that  we  haven't  discussed? 
If  yes,  please  explain. 

No. 


A 
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Questionnaire  2A:  This  questionnaire  was  designed  for  the  Special  Team  members  at  the  completion 
the  plant-specific  plan. 


Plant  had  draft  plan  ready  for  review  at  the  On-site  visit 


Q-2,  When  was  the  plant-specific  HACCP  plan  completed ?_  __May  28, 1992 


Q-3.  Were  there  any  other  Special  Team  activities  that  occurred  after  the  On-Site  Visit  concerning 

development  of  a  planl-spmfic  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes, 
please  specify 


No, 


Q-3 
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Questionnaire  2B:  This  questionnaire  was  designed  for  plant  management  at  the  completion  of  the  plant- 
specific  plan. 


pla  |Q-1«  Since  the  On-Site  Visit,  what  activities  were  undertaken  to  develop  the  plant-specific  model 

(e.g.,  meetings,  phone  calls,  training  sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics 
were  covered  and  what  was  accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was 
there  preparation  time  for  the  activities?  If  so,  how  much  time? 


According  to  the  plant  Quality  Control  Manager,  the  plant  specific  plan  was  developed  at  the  time  of 
the  on-site  visit  Therefore,  this  questionnaire  is  N/A. 


2-2.  Since  the  On-Site  Visit  have  there  been  any  changes  in  the  numbers  or  types  staff  (titles)  who  worked  on 
the  development  of  a  plant-specific  HACCP  plan  based  on  the  generic  HACCP  model?  If  yes,  please 
specify. 


No. 


Q-3.  Were  there  any  other  pilot  plant  activities  that  occurred  after  the  On-Site  Visit  concerning  development  of 
a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes,  please  explain. 


No. 


AI 
■1 
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Questionnaire  1A:  This  questionnaire  was  designed  to  measure  Special  Team  activities  during  the  On- 
Site  Visit. 


PART  1:    Applicability  of  Generic  HACCP  Model 

Q-l.  What  activities  did  you  perform  to  determine  the  applicability  of  the  Generic  HACCP  Model  to  this  plant? 
Please  specify  activity  (e.g.,  used  checklists,  toured  plant,  met  with  various  people,  etc.)  and  the  amount  of 
time  expended. 

The  On-site  was  held  May  26-28, 1992.  A  group  of  24  plant  and  FSIS  personnel  met  first  at  the  hotel 
for  on-site  awareness,  then  the  group  went  to  the  plant  for  a  walk  through  to  match  the  flow  diagram 
developed  at  the  On-Site  to  the  plant  process.  During  the  plant  walk  through,  the  Special  Team  noted 
that  the  plant-specific  plan  discussed  at  the  hotel  was  different  from  the  actual  plant  operations;  steps 
were  out  of  order. 

The  Special  Team  met  with  the  day  shift  inspectors,  the  area  union  representative  and  some  plant 
people  after  the  first  shift  for  a  HACCP  question  and  answer  session.  Initially,  the  group  was 
unwilling  to  ask  questions,  but  after  the  HACCP  slide  presentation,  they  asked  questions  including 
some  like,  "Why  are  you  spending  money  doing  a  HACCP  pilot  program  when  the  agency  is  short  of 
inspectors,  and  how  much  is  this  costing?  At  8:30  pm  after  the  second  shift,  the  Special  Team  met  with 
three  nightshift  inspectors  and  two  plant  people  for  a  similar  session. 

Q-2.  Based  on  the  On-Site  Visit,  was  the  generic  model  applicable  to  this  plant? 

 X  Yes,  Completely  Applicable   Yes,  Somewhat  Applicable   No,  Not  At  All  Applicable 

(Skip  to  PART  2)  (Continue  with  Q-3)  (Continue  with  Q-3) 

Q-3.  What  were  the  difficulties/problems  encountered?  Please  specify. 

No  difficulties. 

Q-4.  If  you  marked  Somewhat  or  No  in  Q-2,  whom  did  you  contact  about  the  difficulties /problems? 
(Check  all  that  apply.) 

HACCP  Special  Team  On-Site  Coordinator 

HACCP  Special  Team  Director 
 Other  HACCP  Special  Team  Members):  (name(s))  

Other  FSIS  Personnel  (EC,  Area,  Regional,  personnel,  etc.) 
 Plant  Personnel  (Management,  QC /Production,  etc.) 

Other:  (title(s)/name(s))   

Q-5.  What  activities  were  undertaken  to  remedy  these  difficulties /problems  (meetings,  phone  calls,  etc.)? 
Please  specify  the  types  and  extent  of  contact  with  each  of  the  people/groups  marked  in  Q-4. 


N/A 
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Q-6.  To  address  the  difficulties  noted  in  Q-3,  did  the  pilot  plant  make  any  changes  to  their  processes/ 
procedures  to  enable  them  to  better  fit  the  Generic  HACCP  Model? 

 Y es  No  (Continue  with  C 

N/A 


Q-7  To  address  the  difficulties  noted  in  Q-3,  were  any  changes  made  to  the  Generic  HACCP  Model? 

N/A   Yes  No  (Continue  with  £ 

Q-8.  By  what  date  were  all  changes  incorporated  into  the  generic  model?  _N\A  , 

PART  2:     Plant-Specific  HACCP  Plan 


Q 


Q-9.  Did  the  pilot  plant  have  an  acceptable  plant-specific  plan  in  place  at  the  start  of  the  On  -Site  Visit? 

Yes,  Completely  Acceptable   Yes,  Partially  Acceptable    „NO__No  Plan/Unacceptable  1 

(Skip  to  Q-11)  (Continue  with  Q-10)  (Continue  with  Q-10) 

Q-lO.If  you  marked  Partially  Acceptable  or  No/Unacceptable  in  Q-9,  what  did  you  do  to  develop  or  initiate 
development  of  a  plant-specific  HACCP  plan  (meetings/phone  calls,  etc.:  how  many,  how  long, 
with  whom,  etc.)? 


Q-11.  Was  a  plant-specific  HACCP  plan  completed  by  the  end  of  the  On-Site  Visit? 

 _Yes  (Continue  with  Q-12)  X  No 

If  no,  please  specify  what  remains  to  be  done  and  when  you  anticipate  such  a  plan  to  be  completed  anc 
ready  for  implementation. 


Q-12.  Were  there  any  other  Special  Team  activities  that  occurred  during  the  On-site  Visit  concerning 

development  of  a  plant  specific  plan  based  on  the  genetic  model  that  we  haven't  discussed?  If  yes, 
please  explain. 

During  the  session  on  developing  CCPs,  tine  Special  Team  broke  the  participants  into  two  groups,  e> 
a  mixture  of  FSIS  and  plant  personnel  with  the  spokesperson  for  each  group  being  a  plant  person.  1 
group  had  the  responsibility  of  developing  all  parts  of  a  CCP  and  creating  an  overhead.  The  Specia 
Team  could  hear  by  their  discussons  that  each  group  understood  the  difference  between  making  up 
generic  CCP  and  developing  a  specific  one  for  their  plant. 

There  was  a  problem  with  the  plant  The  company  w&s  upset  about  collecting  birds  for  samples. 
Apparently,  the  corporate  quality  control  manager  who  was  on  the  HACCP  steering  committee 
volunteered  his  company  to  do  a  pilot  and  selected  this  plant  when  there  were  no  volunteers.  HAC 
material  had  gone  to  corporate  headquarters  and  apparently,  neither  he  nor  the  plant  had  read  all  th 
was  involved  in  participating  in  a  pilot  Finally,  the  company  agreed  to  wait  and  reconsider  its  deci 
to  pull  out  Company  management  called  back  a  week  later  and  agreed  to  follow  all  conditions.  TI 
area  supervisor  handled  the  situation  well  and  play  down  their  anxieties.  Plant  management  was 
anxious  because  of  the  Atlanta  Journal  publicity.  There  was  a  split  within  the  company  between  th 
who  wanted  to  participate  and  those  who  didn't 
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Questionnaire  IB:  T/zi's  questionnaire  was  designed  to  assess  the  time  and  effort  of  pilot  plants  in  develop- 
ing plant-specific  plans  prior  to  and  during  the  On-Site  Visit. 


2-1.  Who  on  your  staff  worked  on  the  adaptation  of  the  generic  HACCP  model  to  a  plant-specific  HACCP  plan? 
(titles/numbers  of  people  involved,  etc.) 

HACCP  Quality  Control  Manager. 


2-2.  What  activities  were  undertaken  to  develop  the  plant-specific  plan  (e.g.,  meetings,  phone  calls,  training 
sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics  were  covered  and  what  was 
accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was  there  preparation  time  for  the 
activities?  If  so,  how  much  time? 

The  Quality  Control  Manager  said  he  had  spent  50  hours  writing  the  plant-specific  plan.  This  time 
estimate  included  the  time  he  spent  consulting  with  technical  experts. 


2-3.  Were  there  any  other  pilot  plant  activities  concerning  the  development  of  a  plant-specific  plan  based  on 
the  generic  model  that  occurred  prior  to  or  during  the  On-Site  Visit  that  we  haven't  discussed? 
If  yes,  please  explain. 

No. 
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Questionnaire  2A:  This  questionnaire  was  designed  for  the  Special  Team  members  at  the  completion 
the  plant-specific  plan 


Q-l,  Since  the  On-Site  Visit,  what  did  you  do  to  continue  the  development  of  the  plant-specific  HACCP  pla 
Please  be  specific  (meetings,  phone  calls,  etc.:  how  many,  how  long,  with  whom). 


One  hour  with  plant  personnel  May  26, 1992s  twelve  hours  with  plant  August  3-4, 1992. 

Phone  calls  approximately  five  hours  total  time.  Calls  were  with  plant  HACCP  Coordinator  and  otl 
plant  personnel. 

Q-2,  When  was  the  plant-specific  HACCP  plan  completed   October  18, 1992  


Q-3.  Were  there  any  other  Special  Team  activities  that  occurred  alter  the  On-Site  Visit  concerning 

development  of  a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes, 
please  specify, 


The  plant-specific  plan  completed  on  October  18, 1992,  will  need  to  be  revised  before  the  plant  can 
proceed  to  Phase  HI  of  the  HACCP  pilot  test  Target  date  for  the  revision  is  February  1993. 


Q-3. 
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Questionnaire  2B:  This  questionnaire  was  designed  for  plant  management  at  the  completion  of  the  plant- 
specific  plan. 


J  Q-l.  Since  the  On-Site  Visit,  what  activities  were  undertaken  to  develop  the  plant-specific  model 

(e.g.,  meetings,  phone  calls,  training  sessions,  etc.)?  What  was  the  purpose  of  each  activity?  What  topics 
were  covered  and  what  was  accomplished  by  each  activity?  How  long  were  the  meetings,  etc.?  Was 
mere  preparation  time  for  the  activities?  If  so,  how  much  time? 

See  Questionnaire  IB. 


2-2.  Since  the  On-Site  Visit,  have  there  been  any  changes  in  the  numbers  or  types  staff  (titles)  who  worked  on 
the  development  of  a  plant-specific  HACCP  plan  based  on  the  generic  HACCP  model?  If  yes,  please 
specify. 


No. 


3-3.  Were  there  any  other  pilot  plant  activities  that  occurred  after  the  On-Site  Visit  concerning  development  of 
a  plant-specific  plan  based  on  the  generic  model  that  we  haven't  discussed?  If  yes,  please  explain. 


No. 


|  III  ||  |    |  I 
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